Keywords: Machine Learning and Data Mining
Abstract: This talk will provide an overview of a personal perspective on inference and learning for graphical models, one that began with work on multi-resolution models for signals and images but that has evolved into a more general look at inference and learning especially for graphical models for which these tasks are tractable and scalable to large problems.

The talk will begin with a brief introduction to Markov models on undirected graphs and message-passing algorithms, often known as Belief Propagation, that exactly solve inference problems for s on a very special set of graphs, namely those without loops or cycles, i.e., trees. We’ll then turn to building or learning models on such graphs, including ones that explicitly have hierarchical structure and will comment on some of the differences between the questions that have typically been addressed in very different communities (namely machine learning and system theory). We’ll then provide a new method for learning models on trees with hidden nodes.

The rest of the talk will deal with a look at what happens if one considers graphs with loops. We first look at what is known as Loopy Belief Propagation and provide, for the Gaussian case, an explicit picture of what it does and when and why it works and when it doesn’t based on what we call walk-sum analysis. We then use these ideas to describe another new set of algorithms based on the graph-theoretic concept of a feedback vertex set (i.e., a set of nodes in the graph that, if removed, leave a cycle-free graph). As time allows we’ll discuss the learning of several other classes of graphical models, where in each case, the objective is to learn models for which both the learning of these models as well as exact or nearly exact inference using these models is computational feasible.

Keywords: Nonlinear System Identification, Neural Networks
Abstract: This work presents the application of an initialization scheme for nonlinear state-space models on a real data benchmark example: the Silverbox problem. The goal of the proposed approach is to transform the identification of a nonlinear dynamic system into an approximate static problem, so that system dynamics and nonlinear terms are identified separately. Classic identification techniques are used to handle dynamics, while regression methods from the statistical learning community are introduced to estimate the nonlinearities in the model. Results obtained on the Silverbox problem are discussed and compared with the performance of other related methods.

Keywords: Nonlinear System Identification, Machine Learning and Data Mining, Basis Functions
Abstract: A novel method, combining Support Vector Machines and Least Squares Prediction Error techniques, for the identification of the linear and nonlinear blocks in a Wiener model is presented in this paper. The identification is carried out by minimizing an augmented cost function defined as the sum of the standard structural risk function appearing in Support Vector Regression and the quadratic criterion on the prediction errors associated to Least Squares estimation methods. The properties of the proposed method are illustrated through simulation examples.

Keywords: Neural Networks, Nonlinear System Identification
Abstract: Abstract: This paper proposes a two-stage orthogonal least squares (OLS) algorithm based on the classical Gram-Schimdt (CGS) method for the construction of wavelet networks. The main objective is to improve the compactness of the wavelet networks model built by the orthogonal forward stepwise methods. The proposed two stage stepwise method selects model terms one by one from a candidate term pool to construct an initial model in the first stage, and then replaces some insignificant terms by reviewing their contributions to the cost function in the second stage, leading to a significantly improved compact model. The efficacy and effectiveness of the proposed technique is illustrated by a numerical example.

Keywords: Nonlinear System Identification, Subspace Methods, Mechanical and Aerospace
Abstract: This paper introduces a new frequency-domain subspace-based method for the identification of nonlinear mechanical systems. The technique exploits frequency data and interprets nonlinearities as feedback forces exciting the underlying linear system. It is demonstrated using two academic examples, a Duffing oscillator and a five degree-of-freedom system comprising two nonlinearities.

Keywords: Nonlinear System Identification, Identification for Control, Continuous Time System Estimation
Abstract: The paper focuses on system identification of a temperature model of an industrial air handling unit in connection with an air conditioned manufacturing zone. The models obtained are intended to be used for a subsequent control analysis and tuning of the presently installed temperature control system. Model structures are predetermined based on simplified first principles modelling approach. A bilinear parametrisation method in tandem with a simplified instrumental variable technique is applied for the parameter estimation of a discrete-time Hammerstein-bilinear model representing the air handling unit. Whilst a simplified instrumental variable method for continuous-time system identification is utilised for estimating the parameters of the linear manufacturing zone model.

Keywords: Continuous Time System Estimation, Model Validation, Multivariable System Identification
Abstract: In this paper, a group of identification methods are closely examined and evaluated with respect to the parameters estimation of polynomial models of a 15 kW hydraulic pumping system. The aim is to determine models with good performance in both transient and steady-state regimes. The linear methods based on parametric model structures cover autoregressive (AR), autoregressive with exogenous variables (ARX), autoregressive with moving average and exogenous variables (ARMAX), Box-Jenkins (BJ) and state-space schemes. The nonlinear method is a nonlinear autoregressive with moving average and exogenous variables (NARMAX) which uses free-run simulation. Through numerical simulations, it is concluded that NARMAX yields the best fit amongst the selected schemes.

Keywords: Input and Excitation Design, Biological Systems
Abstract: The paper presents a methodology for optimal input design for model discrimination from experimental data. To allow analytical solutions, the method, using Pontryagin’s maximum principle, is initially developed for non-linear single state systems that are affine in their joint input. The method is demonstrated on a fed-batch reactor case study with zero-, first-order and Monod kinetics.

Keywords: Vibration and Modal Analysis, Input and Excitation Design
Abstract: The paper proposes an algorithm for the rejection of periodic disturbances of unknown frequency. The frequencies of the components of the disturbance are estimated using an orthogonal signals generator based on second-order generalized integrators. A novel feature of the algorithm is that internal signals of the frequency estimator are combined linearly to produce the control signal, resulting in a very simple algorithm. For a sinusoidal disturbance, the entire scheme is governed by a single adaptive parameter. It is shown that, within the assumptions of an averaging analysis, the adaptive system is globally stable and completely rejects the disturbance, even in the presence of plant uncertainty. Bounds on the uncertainty can be computed a priori based on bounds on the disturbance frequency. Simulations demonstrate the properties of the algorithm in a variety of conditions.

Keywords: Input and Excitation Design
Abstract: Robust optimal experiment design for dynamic system identification is cast as a min-max optimization problem, which is infinite-dimensional. If the input spectrum is discretized (either by considering a Riemmann approximation, or by restricting it to the span of a finite dimensional linear space), this problem falls within the class of semi-infinite convex programs. One approach to this optimization problem of infinite constraints is the so called ``scenario approach'', which is based on a probabilistic description of the uncertainty to deliver a finite program that attempts to approximate the optimal solution with a prescribed probability. In this paper, we propose as an alternative an exchange algorithm based on some recent advances in the field of semi-infinite programming to tackle the same problem. This method is compared with the scenario approach both from the aspects of accuracy and computational efficiency. Furthermore, the comparison includes the MATLAB semi-infinite solver fseminf to provide a general palette of methods approximating the robust optimal design problem.

Keywords: Input and Excitation Design, Biological Systems, Bayesian Methods
Abstract: Identification of stochastic reaction dynamics inside the cell is hampered by the low-dimensional readouts available with today's measurement technologies. Moreover, such processes are poorly excited by standard experimental protocols, making identification even more ill-posed. Recent technological advances provide means to design and apply complex extra-cellular stimuli. Based on an information-theoretic setting we present novel Monte Carlo sampling techniques to determine optimal temporal excitation profiles for such stochastic processes. We give a new result for the controlled birth-death process and provide a proof of principle by considering a simple model of regulated gene expression.

Keywords: Maximum Likelihood Methods, Frequency Domain Identification, Mechanical and Aerospace
Abstract: The problem of parameter estimation with both time and frequency-domain data is considered and a maximum likelihood approach to the problem is derived. In addition, a procedure to perform a coordinated input design in the frequency and time domain is proposed. This procedure is applied to the identification of the parameters of a model for the dynamics of a rotor blade, showing how the proposed co-design allows to achieve sensibly higher accuracy in the estimation of the system parameters.

Keywords: Input and Excitation Design, Maximum Likelihood Methods
Abstract: The Prediction Error Method (PEM) is related to an optimization problem built on input/output data collected from the system to be identified. It is often hard to find the global solution of this optimization problem because the corresponding objective function presents local minima and/or the search space is constrained to a nonconvex set. The existence of local minima, and hence the difficulty in solving the optimization, depends mainly on the experimental conditions, more specifically on the spectrum of the input/output data collected from the system. It is therefore possible to avoid the existence of local minima by properly choosing the spectrum of the input; in this paper we show how to perform this choice. We present sufficient conditions for the convergence of PEM to the global minimum and from these conditions we derive two approaches to avoid the existence of nonglobal minima. We present the application of one of these two approaches to a case study where standard identification toolboxes tend to get trapped in nonglobal minima.

Keywords: Continuous Time System Estimation, Frequency Domain Identification
Abstract: We present an algebraic method to compute a globally optimal H2 approximation of order N-3 to a given system of order N. First, the problem is formulated as a two-parameter polynomial eigenvalue problem with a special structure. To solve it, we apply and generalize algebraic techniques used in the computation of the Kronecker canonical form of a matrix pencil. Finiteness of the number of nontrivial solutions then allows the problem to be reduced to a one-parameter polynomial eigenvalue problem, which is solved with standard numerical methods. An example demonstrates the approach and provides a proof of principle.

Keywords: Others
Abstract: In this paper a solution approach for parametric H₂ model reduction based on Groebner bases is presented. The approach utilizes a naive parametrization of the approximant, which leads to a rather simple set of algebraic equations that is solved by means of Groebner bases. This allows one to deal with the parametric case, and analysis of the optimal solution in the presence of parameters becomes possible. Numerical examples show that the optimal approximant may change discontinuously as a parameter value varies, also showing the existence of systems with two optimal approximants.

Keywords: Identifiability, Continuous Time System Estimation
Abstract: An Exponential-Polynomial-Trigonometric (EPT) function is defined on [0,infty) by its minimal realization (A,b,c). A stable non-negative EPT function of a fixed degree is fitted to a large set of data. Excluding the non-negativity constraint this is shown to be equivalent to a discrete time rational approximation problem which is approached using the RARL2 software. We examine whether the resulting EPT function is non-negative. If the EPT function assumes negative values we then impose non-negativity by optimally adjusting b while keeping the pair (A,c) fixed.

Keywords: Errors in Variables Identification, Subspace Methods, Maximum Likelihood Methods
Abstract: Many problems of system identification, model reduction and signal processing can be posed and solved as a structured low-rank approximation problem. In this paper a reformulation of the structured low-rank approximation problem as minimization of a multivariate rational cost function is considered. We show that in two different parametrizations the problem is reduced to optimization on a compact manifold or to a set of optimization problems on bounded domains of Euclidean space. We make a review of polynomial algebra methods for global optimization of the rational cost function.

Keywords: Continuous Time System Estimation, Frequency Domain Identification
Abstract: The long-standing open problem about whether the number of critical points in the H2 SISO real model order reduction problem is finite is answered in the positive in the case the transfer function of the to-be-reduced model has distinct poles (ie. only has poles of algebraic multiplicity one). This has important implications for various search methods for finding critical points or local minima of the criterion function for this model reduction problem. In fact more is shown namely that in a particular parametrization the number of complex solutions of the first order conditions of the H2 real model order reduction problem is finite and lies in a bounded set of which the bound can be determined from information about the to-be-reduced model. This implies that the H2 model order reduction problem can be solved in principle by constructive algebra methods (such as Groebner basis methods) in case the to-be-reduced model has distinct poles. It simplifies the methods for co-order three reduction as described in a companion paper.

Keywords: Multivariable System Identification
Abstract: We consider the problem of approximating a linear time-varying pxm discrete-time state space model S of high dimension by another linear time-varying pxm discrete-time state space model of much smaller dimension, using an error criterion defined over a finite time interval. We derive the gradients of the norm of the approximation error and show how this can be solved via a fixed point iteration.

Keywords: Mechanical and Aerospace, Machine Learning and Data Mining, Nonlinear System Identification
Abstract: In this paper, a novel approach for NOx generation modeling in heavy-duty (HD) Diesel engines is proposed. Only the indicated pressure and the speed measurements are used, standing on the assumption that all combustion phenomenona are reflected by the crank angle resolved pressure trajectory. A principal component analysis is performed to describe the information included in the pressure by means of a limited set of variables; this enables the use of simple identification techniques to derive a simple and reliable predictor, also suited for on-line estimation. The proposed strategy is implemented on a off-road HD Diesel engine and validated on a standard test cycle.

Keywords: Model Validation, Nonlinear System Identification
Abstract: LNT and SCR are two leading candidates for Diesel exhaust nitrogen oxide (NOx) after-treatment. This paper deals with the modeling of the architecture combining the two systems in series with NO/NO2 differentiation, induced directly by the widening and hardening of the future standards. Model reduction is performed to allow for real-time automotive applications. Based on simplified chemistry and slow-fast dynamics assumptions, a complete reduced model is proposed, suitable for on-board diagnosis and model-based control. Validation has been achieved through extensive experiments.

Keywords: Other, Bayesian Methods, Identification for Control
Abstract: The paper deals with a problem of fuel consumption optimization. Solutions existing in this field are mainly based on the various conceptual approaches such as hybrid and electric vehicles. However, it leads to high initial cost of a vehicle. The approach presented in this paper aims at conventional vehicles and is based on recursive algorithms of system identification and adaptive quadratic optimal control under Bayesian methodology. Experiments with real data measured on a driven vehicle are provided.

Keywords: Identification for Control, Filtering and Smoothing
Abstract: This paper deals with a dynamic estimator for fully automated parameters indentification of permanent magnet three-phase synchronous motors. High performance application of permanent magnet synchronous motors (PMSM) is increasing. PMSM models with accurate parameters are significant not only for precise control system designs but also in traction applications. Acquisition of these parameters during motor operations is a challenging task due to the inherent nonlinearity of motor dynamics. This paper proposes parameters estimator technique for PMSMs. The technique uses a decoupling procedure optimized by a minimum variance error to estimate the inductance and resistance of the motor. Moreover, a dynamic estimator is shown. The estimator uses the measurements of input voltage, current and mechanical angular velocity of the motor, the estimated winding inductance, and resistance to identify the amplitude of the linkage flux. The presented technique is generally applicable and could be used also for the estimation of mechanical load and for other types of electrical motors, as well as for other dynamic systems with nonlinear model structure. Through simulations of a synchronous motor used in automotive applications, this paper verifies the effectiveness of the proposed method in identification of PMSM model parameters and discusses the limits of the found theoretical and the simulation results.

Keywords: Nonlinear System Identification, Identification for Control, Model Validation
Abstract: The paper presents a new procedure to identify at the same time the electrical parameters of a permanent magnet stepper motor (PMSM) and the initial offset of an incremental encoder. The model considers effects due to the permanent magnet and to variable reluctance, and the resulting theory is applicable to cases where both or only one of these terms is present. The standard DQ model of PMSM's assumes that the permanent magnet is lined up with a winding when the position is zero. When an incremental encoder is used, an initialization procedure is required to zero the initial offset. In contrast, this paper computes a transformed model that accounts for the initial offset. then develops a least-squares identification algorithm that estimates the machine's electrical parameters together with the offset angle. Experiments show that the estimation procedure and a closed-control method using the estimated offset perform similarly compared to when the offset is reset to zero using an initilization procedure.

Keywords: Mechanical and Aerospace, Grey Box Modelling, Identification for Control
Abstract: Recent technology advances in the field of ride-by-wire technology for motorcycle (namely active braking and full electronic throttle) open the way to the design of innovative control strategies to improve two-wheeled vehicles stability. As such, it is of growing importance to devise control oriented models of the bike dynamics to be employed for control design purposes. This paper proposes an analytical model of a two-wheeled vehicle tuned to capture the coupling between longitudinal variables (i.e. traction and braking torque) and out-of-plane modes. The model is derived from first principles. The model parameters are identified from a complete multi-body simulator. The proposed model offers a good tradeoff between complexity and accuracy.

Keywords: Other, Identifiability, Biological Systems
Abstract: This work addresses the problem of identifying hypnotic models of patients undergoing deep sedation for ultrasonographic endoscopy. The model used assumes that there are three inputs and one output. The output is the level of hypnosis. Two of the inputs correspond to manipulated variables and are given by the perfusion rates of the hypnotic drug (propofol) and analgesic drug (remifentanil). In addition to these input signals that are known, there is another input that corresponds to a disturbance and may not be measured. This unmeasurable input is related to the noxious stimuli applied to the patient due to the manoeuvres of the endoscopic device. In order to take into account the unmeasurable input, a bi-phase estimation procedure is proposed. In the first phase, an intermediate model with just the measurable inputs is identified. Large average discrepancies between the intermediate model and the clinical data are then used to estimate the stimulus input. In the second phase, the model relating the measurable inputs and the output is identified again, discounting from output data the estimated stimulus input. Before the identification, a local identifiability analysis is performed that allows to decide which parameters are to be estimated from data and which are the ones whose values should be {em a priori} selected based on previous insight. The contribution of the paper consists in the bi-phase algorithm used to identify systems with unknown inputs and to demonstrate the identification procedure proposed using actual clinical data from 20 patients.

Keywords: Other, Nonlinear System Identification, Model Validation
Abstract: Depth of anesthesia is usually quantified by the Bispectral Index (BIS) and refers to both loss of consciousness, resulting from the administration of a hypnotic like propofol, and inhibition of pain, resulting from the administration of an analgesic like remifentanil. This paper addresses the mathematical modeling of the joint effect of propofol and remifentanil in the depth of anesthesia, using BIS measurements. Two models and identification strategies are considered. The first model is based on standard pharmacokinetic/pharmacodynamic models and the associated identification strategy corresponds to the application of a hybrid method. The second model has a minimal number of parameters and the associated identification strategy corresponds to the application of a prediction error method. These two approaches are tested and compared on real data.

Keywords: Biological Systems, Other, Continuous Time System Estimation
Abstract: This paper examines the use of system identification to describe time-varying phenomena in a smoking cessation intervention. The analysis is facilitated by the availability of intensive longitudinal data that enables the application of system identification techniques. Two model structures are considered; one involves the concept of statistical mediation, while the other describes a feedback mechanism. In fitting these models to intensive longitudinal data from a University of Wisconsin clinical trial that studied bupropion and counseling as smoking cessation aids, we focus on the relationship between craving and smoking. Here, we find craving features inverse response and smoking behavior features a dramatic reduction on the quit date, followed by a resumption in smoking. Analyzing the resulting models, we find that they differ in how they describe smoking resumption, and the case is made that the feedback mechanism more appropriately describes the relationship between craving and smoking.

Keywords: Biological Systems, Basis Functions
Abstract: The cerebrovascular bed is very sensitive to CO2 changes, particularly in respiratory-related areas, such as the brainstem. Therefore, the hemodynamic response to such changes is of interest as it quantifies this sensitivity. Here, we examine in detail the regional characteristics of the hemodynamic response to spontaneous and larger, externally induced step CO2 changes CO2 (end-tidal forcing) by utilizing BOLD functional magnetic resonance imaging (fMRI) measurements from healthy humans. We first obtain estimates of the impulse response between CO2 and BOLD signal in several anatomically and functionally defined regions of interest, using function expansions with different basis sets. These include the Laguerre basis, which has been widely used in linear and nonlinear systems identification particularly for biological/physiological systems, as well as different variants of gamma functions, which have been widely used in functional neuroimaging due to physiological considerations with regards to the characteristics of the BOLD response to external (sensory or other) stimuli. Based on the aforementioned comparisons, we perform the same analysis in smaller anatomical areas, considering voxel neighborhoods that span the entire image, in order to map key features of the hemodynamic response function such as peak value, time-to-peak and area, in finer spatial resolution.

Keywords: Nonlinear System Identification, Biological Systems, Subspace Methods
Abstract: It is not trivial to acquire data under stationary conditions from biomedical systems since they frequently show time-varying and/or switching behaviour. It is often possible to acquire short segments of stationary data and repeat the experiment many times. However, initial conditions contribute substantially to the transient response and must therefore be accounted for explicitly. This paper presents a subspace algorithm for the identification of Hammerstein systems from short segments of data that estimates the initial condition of each segment and the parameters of the nonlinearity, as well as a state-space model for the linear part. A previously developed algorithm suffers from two issues. Firstly, all segments had to be equal lengths, and secondly the algorithm provided an over-parameterized model of the Hammerstein system rather than an individual model for each component of the cascade. We resolved the first issue by introducing a new formulation of the problem and the second one by developing an iterative method to separate the estimated parameters. Simulation results on Hammerstein model of reflex joint stiffness show the algorithm is capable of identifying accurate models even with noisy data. We also show the application of this algorithm on a set of experimental data acquired from one subject.

Keywords: Biological Systems, Nonlinear System Identification, Grey Box Modelling
Abstract: The Major Depressive Disorder (MDD) represents one of the major public rising health care problems worldwide according to the World Health Organization. In this work we introduce a novel dynamic model for the analysis of clinimetric data obtained from the follow-up assesment in depressed patients, and the identification of its parameters and analysis of its equilibria and stability, that also helps to evaluate the effectiveness of some given treatments. The resulting reduced model is a positive nonlinear system with two parameters to identify the intensity of the treatment and the dynamics of the depression levels in a Hamilton Rating Scale for Depression. In order to verify the validity of the the model, a group of 39 participants diagnosed with MDD meeting DSM-IV criteria was divided in four groups, including a controlled clinical trial, randomly assigned to one of four treatment options: a) repetitive transcranial magnetic stimulation (rTMS) + escitalopram; b) rTMS+ placebo, c) sham rTMS + escitalopram and sham rTMS and placebo. Clinical assesments were done weekly using HDRS as main outcome measure in order to identify the parameters of the model using a Least-Squares algorithm. The results show that the intensity and the dynamics of clinical depression can be established with a simple nonlinear model.

Keywords: Continuous Time System Estimation, Other, Process Control
Abstract: Direct identification of continuous-time models from sampled data is now mature. The developed methods have proven successful in many practical applications and are available as user-friendly and computationally efficient algorithms in the CAPTAIN and CONTSID toolboxes for Matlab. Surprisingly many practitioners appear unaware that such methods not only exist but may be better suited to their modelling problems. This paper discusses and illustrates with the help of real-life data the advantages of these direct schemes to continuous-time model identification.

Keywords: Mechanical and Aerospace, Continuous Time System Estimation, Subspace Methods
Abstract: Accurate dynamic modelling of helicopter aeromechanics is becoming increasingly important, as progressively stringent requirements are being imposed on rotorcraft control systems. System identification plays an important role as an effective approach to the problem of deriving or fine tuning mathematical models for purposes such as handling qualities assessment and control system design. In this paper the problem of deriving continuous-time models for the dynamics of a small-scale quadrotor helicopter is considered. More precisely, the continuous-time predictor-based subspace identification approach is adopted and the results obtained in an experimental study are presented and discussed.

Keywords: Identification for Control, Multivariable System Identification, Recursive Identification
Abstract: The paper is concerned with the practical aspects of a unified approach to the identification and estimation of multiple-input, single-output (MISO) transfer function models for both continuous and discrete-time systems. The estimation algorithms considered in the paper are based on the Refined Instrumental Variable (RIV) approach to identification and estimation, where the MISO model denominator polynomials are normally constrained to be equal. Unconstrained RIV estimation presents a more difficult problem and it is necessary to exploit an iterative, back-fitting routine to handle this more general situation. The paper focuses on the practical realization of this back-fitting algorithm, including its initiation from either common denominator MISO or repeated SISO estimation. The rivcdd algorithm for continuous-time model estimation, as implemented in the CAPTAIN Toolbox for Matlab is then used in three practical examples: first, the modelling of solute transport and dispersion in a water body; secondly, modelling for two control problems, namely a pair of connected laboratory DC motors and a nonlinear wind turbine simulation.

Keywords: Continuous Time System Estimation, Nonlinear System Identification, Recursive Identification
Abstract: This paper studies the identification problem for a class of Hammerstein-Wiener continuous-time systems, with a monotonic nonlinear function for the Wiener part. Based on the previously developed simplified refined instrumental variable method, and by making use of an adaptive observer for data filtering, a combined approach, referred to as the Kalman pre-filtered instrumental variable based method, is proposed. By taking the advantages of the two aforementioned methods, the new method is faster and has a naturally stabilized filter, as well as keeps a high estimation accuracy in most cases. Monte Carlo simulation analysis is used to illustrate the performances of the proposed methods.

Keywords: Continuous Time System Estimation, Maximum Likelihood Methods, Multivariable System Identification
Abstract: The EM algorithm has been successfully applied to obtain maximum likelihood estimates for state-space models. The usual formulation of the algorithm is based on a shift operator model for the discrete-time (or sampled-data) system. More recently, it has been shown that an equivalent formulation of the algorithm in terms of incremental discrete-time models shows better numerical properties, in particular, for fast sampling rates. In this paper we explore the correspondence between the parameter estimates given by the EM algorithm applied to incremental models and those corresponding to a pure continuous-time formulation.

Keywords: Continuous Time System Estimation
Abstract: This paper presents instrumental variable methods for identifying partial differential equation models of distributed parameter systems in presence of output measurement noise. Two instrumental variable-based techniques are proposed to handle this continuous-time model identification problem: a basic one using input-only instruments and a more sophisticated refined instrumental variable method. Numerical examples are presented to illustrate and compare the performances of the proposed approaches.

Keywords: Closed Loop Identification
Abstract: A new identification algorithm is investigated for direct closed-loop identification by using the cyclostationarity of output over-sampled data. It has been shown that the plant model can be directly identified from the input and output data in the output over-sampling scheme even less excitation is available in the test input. However, the numerical optimization in conventional direct algorithms ordinarily depends on the initial values and estimation of noise process, whereas the estimation accuracy of the noise model is fragile to the poles and zeros of its transfer function. The properties of instinct cyclostationarity and the associated subspace characteristics of the sampled data in the output over-sampling scheme are analyzed in the paper. It illustrates that these properties can be applied for identification, and can reduce the influence of sensitivity to the noise model estimation and initial values in the numerical optimization. The simulation examples illustrate that the proposed algorithm can significantly improve the identification performance in direct closed-loop identification.

Keywords: Closed Loop Identification, Continuous Time System Estimation, Vibration and Modal Analysis
Abstract: This paper deals with joint stiffness identification with a new Closed-Loop Output Error (CLOE) method which minimizes the quadratic error between the actual motor force/torque and the simulated one. This method is based on the DIDIM (Direct and Inverse Identification Model) procedure which has been validated on rigid robots and which is now applied to a flexible joint robot. DIDIM method requires a gain updating in the simulated robot in order to keep the bandwidth of the rigid controlled degree of freedom (dof ) and to keep the natural frequency of the flexible dof, close to the actual ones, at each step of the recursive Gauss Newton non linear programming algorithm. This gain updating requires a first step of estimating the natural frequency of the flexible dof before applying DIDIM method in a second step. An experimental setup exhibits identification results and shows the effectiveness of our approach.

Keywords: Closed Loop Identification, Subspace Methods, Recursive Identification
Abstract: Bias compensated state space model identification method in close loop environment (CL-BCSS) is introduced. The noise is assumed to be a 0 mean colored noise. It is also assumed that the reference input is a white Gaussian process uncorrelated to the noise. Signal and noise components of a certain matrix in the proposed method and CL-MOESP method are anlyzed and compared. It is shown that CL-BCSS method corresponds to CL-MOESP method with infinite past horizon.

Keywords: Closed Loop Identification, Identification for Control
Abstract: This paper presents a new closed-loop identification method for first-order and second-order plus-dead-time models, using first-order Padé approximation. In this method, the influence of the “zero” relative to the Padé approximation is considered in the temporal parameters transient behavior. Simulation results are presented, showing the method performance for some kinds of systems. An experimental platform with a heat sink bar was used to apply this identification method.

Keywords: Closed Loop Identification, Bounded Error Identification
Abstract: The problem of closed loop system identification in presence of bounded disturbances is considered. In this paper two recursive algorithms are proposed to solve this identification problem. Stability and convergence properties are demonstrated. Simulation results validate the proposed solutions.

Keywords: Hybrid and Distributed System Identification, Identifiability, Closed Loop Identification
Abstract: Modeling of dynamical properties of highly complex and interconnected systems becomes important in different fields of science. When identifying the structure and dynamics of a network of interconnected dynamical systems, including cause-effect relations, there is a tendency to use nonparametric or FIR models of the output error type. In this paper it is shown, and illustrated by some simple examples, that appropriate attention should be given to using flexible noise models, in order to allow consistent identification of the dynamics, while the use of external excitation/probing signals may reduce this need. It is a first step towards using prediction error identification tools to identify the structure of a network.

Keywords: Identifiability
Abstract: The last decade has witnessed an increasing interest in applying systems science concepts for problems in behavioral health, and using these to inform the design, analysis, and implementation of optimized interventions. How can system identification and control engineering impact interventions for chronic, relapsing disorders such as drug abuse, cigarette smoking and obesity? The paper addresses this question by focusing on the problem of time-varying "adaptive" interventions. In an adaptive intervention, dosages of intervention components are assigned based on the assessed values of tailoring variables that reflect some outcome measure (e.g., number of cigarettes smoked, parental function) or adherence (e.g, days abstinent). Because time-varying adaptive interventions constitute closed-loop dynamical systems, they are correspondlngly amenable to control engineering solutions. System identification is enabled by intensive longitudinal data (ILD) that can be obtained in the field via ecological momentary assessment (EMA); this creates the availability of rapidly sampled, continuous-time assessments from which dynamical system behavior can be discerned and modeled. How can system identification and control be applied in this broad setting is demonstrated with a number of illustrative problems: dynamic modeling and hybrid model predictive control of low-dose naltrexone as treatment for fibromyalgia, a chronic pain condition; modeling of a smoking cessation intervention involving bupropion and counseling; constructing a dynamic model of an intervention for preventing excessive weight gain during pregnancy, and Model-on-Demand Model Predictive Control in a hypothetical intervention based on the Fast Track program for assigning the frequency of home counseling visits to families with at-risk children.

Keywords: Filtering and Smoothing, Maximum Likelihood Methods, Machine Learning and Data Mining
Abstract: Kalman smoothers reconstruct the state of a dynamical system starting from noisy output samples. While the classical estimator relies on quadratic penalization of process deviations and measurement errors, extensions that exploit Piecewise Linear Quadratic (PLQ) penalties have been recently proposed in the literature. These new formulations include smoothers robust with respect to outliers in the data, and smoothers that keep better track of fast system dynamics, e.g. jumps in the state values. In addition to L2, well known examples of PLQ penalties include the L1, Huber and Vapnik losses. In this paper, we use a dual representation for PLQ penalties to build a statistical modeling framework and a computational theory for Kalman smoothing.

We develop a statistical framework by establishing conditions required to interpret PLQ penalties as negative logs of true probability densities. Then, we present a computational framework, based on interior-point methods, that solves the Kalman smoothing problem with PLQ penalties and maintains the linear complexity in the size of the time series, just as in the L2 case. The framework presented extends the computational efficiency of the Mayne-Fraser and Rauch-Tung-Striebel algorithms to a much broader non-smooth setting, and includes many known robust and sparse smoothers as special cases.

Keywords: Filtering and Smoothing, Nonlinear System Identification, Bayesian Methods
Abstract: In this paper, we propose and demonstrate the direct use of optimization to search for the mode of the joint posterior state distribution of stochastic nonlinear dynamical systems. That is accomplished by forming a very large but sparse nonlinear optimization problem with the states in all time instants as decision variables. The proposed method generalizes well for parameter estimation without the need for treating them as augmented states and the introduction of artificial dynamics. It is also possible to estimate parameters such as the noise variances, which are assumed known in traditional methods.

Keywords: Filtering and Smoothing, Bayesian Methods
Abstract: The paper deals with state estimation of nonlinear stochastic systems with a special focus on the unscented Kalman filter. Recently, several techniques have been proposed to improve estimate quality of the system state by adapting a scaling parameter of the filter. They are, however, tied with an increase of computational costs. To eliminate this drawback a control mechanism is developed in this paper. Its aim is to execute the adaptation only in a case of strongly nonlinear behavior of the measurement function, which is evaluated using two measures of nonlinearity proposed in this paper. To further reduce the computational costs, the paper focuses on a choice of the interval over which the scaling parameter is adapted. The computational costs saving is illustrated in a numerical example.

Keywords: Filtering and Smoothing, Bayesian Methods
Abstract: The problem of noncausal identification of nonstationary, linear stochastic systems, i.e., identification based on prerecorded input/output data, is considered. It is shown how several competing Kalman-type parameter smoothers, differing in smoothness constraints and memory settings, can be combined together to yield a better and more reliable smoothing algorithm. The resulting parallel estimation scheme automatically adjusts its smoothing bandwidth to the unknown, and possibly time-varying, rate of nonstationarity of the identified system. It also allows one to account for parameter jumps and for the distribution of measurement noise.

Keywords: Filtering and Smoothing, Nonlinear System Identification, Other
Abstract: We address the problem of designing a data-driven soft sensor to estimate the downhole pressure in gas-lifted oil wells. Such application is based on a two-step procedure. In the first step, discrete-time black-box models are identified offline using experimental data. In the second step, recursive predictions of these models are combined with current measured data (of variables other than the downhole pressure) by means of an interacting bank of unscented Kalman filters. In doing so, a closed-loop model prediction is performed. Results indicate that the accuracy of the estimated pressure lays between the accuracy of the simple free-run simulation of the dynamic model and the accuracy of the one-step-ahead prediction of such model. Results indicate that such closed-loop scheme improves estimation accuracy compared to the free-run model prediction. The method tested in this paper can also be applied to other soft sensing applications in industry.

Keywords: Filtering and Smoothing
Abstract: This paper considers a continuous-discrete (CD) nonlinear filtering problem by using the divided difference method (Schei 1997, Norgaard 2000, Simandl 2009), and derives CD divided difference first-order (CD-DD1) and second-order (CD-DD2) filters. Numerical integration procedures for the time update differential equations for the conditional mean and covariance matrix are obtained by using both Euler and Heun schemes. By using two simple models, we have performed comparative simulation studies, together with the CD unscented Kalman filter (Sarkka07), to show the feasibility of the CD-DD1 and CD-DD2 filters.

Keywords: Biological Systems, Model Validation
Abstract: Models of intracellular biochemical reaction networks are difficult to parameterise due to the low number of quantitative time series experimental values. Therefore, model validation or invalidation plays an important role, as it allows to check qualitatively whether a model structure is suited or not to reproduce qualitatively the experimental findings.

This paper analyses the robustness of an experimentally validated polynomial differential equation model of TNF-induced pro-and anti-apoptotic signalling. The bistability of the median model is shown robust to large single parameter variations. Only two parameters (XIAP and Procaspase-3 production rates) are shown to be fragile, in particular when changed simultaneously. Therefore, the model seems valid from the point of view of robustness analysis of the bistability.

Many biological experiments quantify average concentrations or the percentage of viable cells, while other methods such as microscopy-based experiments observe single cells. The integration of single cell and cell population behaviour of TNF-induced pro- and anti-apoptotic signalling has been achieved via a cell ensemble model, whose robustness is also analysed here. We show that within the cell population there are cells with not only quantitative differences, but also qualitative ones. In particular, all cells are not bistable. The degree of robustness applicable for the median cell is expanded to combine mono- and bistable models. This measure, applied solely to the two-dimensional subspace of fragile parameters, is shown to correlate well with the time of death. While robustness of bistability can serve for model validation of the median cell model, it cannot for the model of the cell population.

Keywords: Nonlinear System Identification, Bounded Error Identification
Abstract: It is described how set membership identification and model rejection for polynomial models can be described using polynomial inequalities and inequations. Using difference algebra methods these problems can be reduced to a form based on so called autoreduced sets. It is shown that these descriptions generalize state space descriptions. It is also discussed how special forms of autoreduced sets can make calculations based on interval methods easier to implement.

Keywords: Identifiability, Nonlinear System Identification, Biological Systems
Abstract: Ordinary differential equation models often contain a large number of parameters that must be determined from measurements by parameter estimation. For a parameter estimation procedure to be successful, there must be a unique set of parameters that can have produced the measured data. This is not the case if a model is not structurally identifiable with the given set of outputs selected as measurements. We describe the implementation of a recent probabilistic semi-numerical method for testing local structural identifiability based on computing the rank of a numerically instantiated Jacobian matrix (observability/identifiability matrix). To obtain this, matrix parameters and initial conditions are specialized to random integer numbers, inputs are specialized to truncated random integer coefficient power series, and the corresponding output of the state space system is computed in terms of a truncated power series, which then is utilized to calculate the elements of a Jacobian matrix. To reduce the memory requirements and increase the speed of the computations all operations are done modulo a large prime number. The method has been extended to handle parametrized initial conditions and is demonstrated to be capable of handling systems in the order of a hundred state variables and equally many parameters on a standard desktop computer.

Keywords: Nonlinear System Identification, Input and Excitation Design, Biological Systems
Abstract: Dynamic process models are widely used for operating, controlling and optimising important bioprocesses, e.g., pharmaceuticals, enzyme production and brewing. After selection of an appropriate process model structure, parameter estimates have to be obtained based on real-life experiments. To reduce the amount of labour and often cost intensive experiments Optimal Experiment Design (OED) is an indispensable tool. In Optimal Experiment Design for parameter estimation a scalar measure of the Fisher Information Matrix is used as an objective function. Over the years different criteria have been developed. These criteria may be competing as they each have a slightly different objective. For systematically evaluating the competing nature and to improve the parameter estimation procedure, a multi-objective optimisation approach is selected. To solve the multi-objective dynamic optimisation problems efficiently ACADO Multi-Objective (www.acadotoolkit.org) has been employed, which is a flexible toolkit for solving dynamic optimisation or optimal control problems with multiple and conflicting objectives.

Keywords: Nonlinear System Identification, Subspace Methods, Biological Systems
Abstract: System identification of polynomial and of rational systems is inherently an approximation problem. In the area of systems biology and also in the area of engineering, there is a need for system identification of these classes of systems. In this paper,procedures and algorithms for system identification and model reduction of rational systems are proposed. The proposed procedures and algorithms are based on realization theory for rational systems.

Keywords: Machine Learning and Data Mining
Abstract: It is well known that a linear regression can benefit from knowledge that the underlying regression vector is sparse. The combinatorial problem of selecting the nonzero components of this vector can be relaxed by regularizing the squared error with a convex penalty function like the l₁ norm. However, in many applications, additional conditions on the structure of the regression vector and its sparsity pattern are available. Incorporating this information into the learning method may lead to a significant decrease of the estimation error. In this paper, we review a recently proposed family of convex penalty functions, which encode prior knowledge on the structure of the vector formed by the absolute values of the regression coefficients. This family subsumes the l₁ norm and is flexible enough to include different models of sparsity patterns, which are of practical and theoretical importance. We discuss special cases of these penalty functions in which the regularized empirical error function can be efficiently minimized by a proximal-point method. We compare this method to a previous method based on block coordinate descent and present numerical experiments which highlight the benefit of our framework over a greedy approach and the Lasso method.

Keywords: Bayesian Methods, Machine Learning and Data Mining
Abstract: Popular convex approaches for sparse estimation such as Lasso and Multiple Kernel Learning (MKL) can be derived in a Bayesian setting, starting from a particular stochastic model. In problems where groups of variables have to be estimated, we show that the same probabilistic model, under a suitable marginalization, leads to a dierent non-convex estimator where hyperparameters are optimized. Under assumption of orthogonal regressors, mean squared error (MSE) properties, which are independent of the correctness of the priors entering the sparse model, are investigated to clarify the advantages of our nonconvex technique in comparison with MKL and the group version of Lasso.

Keywords: Nonparametric Methods, Machine Learning and Data Mining
Abstract: In this work we are interested in the problems of supervised learning and variable selection when the input-output dependence is described by a nonlinear function depending on a few variables. Our goal is to devise a sparse nonparametric model, avoiding linear or additive models. The key intuition is to measure the importance of each variable in the model by making use of partial derivatives. Based on this idea we propose and study a new regularizer and a corresponding least squares regularization scheme. Using concepts and results from the theory of reproducing kernel Hilbert spaces and proximal methods, we show that the proposed learning algorithm induces a minimization problem which can be provably solved by an iterative procedure. The consistency properties of the obtained estimator are studied both in terms of prediction and selection performance.

Keywords: Basis Functions, Continuous Time System Estimation, Frequency Domain Identification
Abstract: This paper considers the use of sparse estimation techniques to determine an appropriate set of basis functions, in terms of the number of poles and their respective location, to be used in a system identification problem. In particular, the problem cast in the paper is based on the identification of set of parameters to represent a system of large dynamic range. The proposed solution is based on a LASSO-type sparse estimator, which provides an automatic method for selecting both the number of poles and their location. A simulation example is provided that consists of a highly resonant system with eight resonances that extends over 9 decades of frequency.

Keywords: Maximum Likelihood Methods, Machine Learning and Data Mining
Abstract: In many practical situations, it is highly desirable to estimate an accurate mathematical model of a real system using as few parameters as possible. This can be motivated either from appealing to a parsimony principle (Occam's razor) or from the view point of the utilization complexity in terms of control synthesis, prediction, etc. At the same time, the need for an accurate description of the system behavior without knowing its complete dynamical structure often leads to model parameterizations describing a rich set of possible hypotheses; an unavoidable choice, which suggests sparsity of the desired parameter estimate. An elegant way to impose this expectation of sparsity is to estimate the parameters by penalizing the criterion with the l0 norm of the parameters, which is often implemented as solving an optimization program based on a convex relaxation (e.g. l1/LASSO, nuclear norm, ...). However, in order to apply these methods, the (unpenalized) cost function must be convex. This imposes a severe constraint on the types of model structures or estimation methods on which these relaxations can be applied. In this paper, we extend the use of convex relaxation techniques for sparsity to general rational plant model structures estimated by using prediction error minimization. This is done by combining the LASSO and the Steiglitz-McBride approaches. To demonstrate the advantages of the proposed solution an extensive simulation study is provided.

Keywords: Other
Abstract: Large interconnected power systems are arguably some of the most complicated man-made systems to understand and to characterize. The scale of the problem is immense, involving large numbers of generators, controllers, and transmission lines covering millions of square kilometers. Measurement technology has reached a point where Phasor Measurement Units (PMUs) are being widely installed in power systems all over the world. These devices provide time synchronized (via GPS) phasor measurements from throughout the power grid to Phasor Data Concentrators (PDCs) at power system control centers. These time series can be used to better characterize the system and hopefully, in the long term, to better control the system. This paper presents a tutorial on estimating power system characteristics from measured responses. About a given operating point, power system low-frequency dynamics are well modeled as a high-order, multi-input, multi-output linear system. Of primary interest is the estimation of the inter-area electromechanical modes of the system. These inter-area modes involve generators from one area of the system oscillating against generators in another area of the system. The modes are characterized by their frequency, damping, and shape. In August 1996, the western United States experienced a massive wide spread black out caused by an unstable inter-area mode, involving generators in the north swinging against generators in the south. This paper overviews the problem and examines several methods of estimating the electromechanical modes under different signal conditions. Several real-world examples are given for estimating the electromechanical modes from ambient, transient, or probing situations. When the system is probed, more general state-space and transfer-function models are estimated. Probing a power system with known inputs is challenging and is discussed in this paper. Estimation performance issues are also discussed.

Keywords: Vibration and Modal Analysis, Nonparametric Methods, Filtering and Smoothing
Abstract: For power networks such as the Nordic Grid, that have operation constraints limits imposed by the existence of low-damped electromechanical oscillations, the estimation of electromechanical mode properties is of crucial importance for providing power system control room operators with adequate indicators of the stress of their network. This article addresses the practical application of different spectral analysis techniques that can be used for the estimation of electromechanical mode properties using data emerging from real synchronized phasor measument units (PMUs) located at both the low-voltage distribution and high-voltage transmission networks of the Nordic grid. Emphasis is made on providing systematic approaches to deal with imperfect data found in practice so that accurate estimates can be computed.

Keywords: Fault Detection and Diagnosis, Vibration and Modal Analysis, Other
Abstract: Wide-area monitoring systems (WAMS) that utilize synchrophasor measurements are being deployed in power systems worldwide for the establishment of real-time situational awareness and power system control. A power system Frequency Monitoring Network (FNET) was established in 2004 as a pioneering WAMS to serve the North American power grid. This paper provides a primer of the FNET architecture, its state-of-the-art applications and advanced situational awareness techniques, and research to develop measurement-based power system dynamics models. Finally, FNET analysis of an actual power system event is presented.

Keywords: Vibration and Modal Analysis, Multivariable System Identification, Time Series
Abstract: Estimating power system electromechanical modes has been a topic of great interest since an unstable mode caused a system outage in the western United States in 1996. In recent years several methods have been proposed to estimate the modal frequency and damping from measured data. Most of these methods focus on examining one signal at a time. Some of the methods do, however, incorporate several measured signals and thus have the benefit of taking advantage of information from multiple data channels to increase the performance of the algorithms. This paper compares four published methods of multichannel electromechanical modal estimation techniques: the Modified Extended Yule-Walker (MEYW) method, the Frequency Domain Decomposition (FDD), the Numerical Algorithm For System Identification (N4SID), and a Least Squares (LS) estimate of a multichannel transfer function (MCTF). Through the use of simulated power system data, it is shown that the MCTF method displays superior performance, especially in resolving closely spaced modes. The use of probing signals is shown to increase the performance even further.

Keywords: Identification for Control
Abstract: This article describes a way to use standard System Identification techniques to determine all pool properties of open water channels for controller design. These properties are the delay time, the storage area and the frequency and magnitude of the first resonance peak. The identification procedure is tested on an actual open water channel at the Central Arizona Irrigation and Drainage District, Eloy, AZ. The identification experiment results in excellent estimations of all properties. Especially the value for the storage area improves considerably compared to previous research on this specific open water channel.

Keywords: Other, Neural Networks
Abstract: Computational models are used for rainfall-runoff modelling and forecasting. These models can be broadly categorized as physically-based models, statistical models and data-driven models. Physically-based rely on mathematical equations describing the physical laws of mass and momentum conservation. Statistical models use time series analysis techniques to make predictions and data driven models employ modern computational intelligence tools. Although these models are often used interchangeably in rainfall-runoff modelling and forecasting applications, factors such as data requirements and model complexity inherent in each of these modelling paradigms may render certain models more amenable to modelling applications and others to forecasting applications. The motivation for this paper is thus to elucidate the differences between the modelling paradigms and suggest the suitability of these models in the context of flow forecasting applications. This study reviewed pertinent features between physically-based, statistical and data driven models that are commonly used in flow forecasting. Model results over vastly spatial scales were compared in the light of the differences between modelling paradigms. For the two case studies presented, the physically-based models in general fared worse than data driven models. This is attributed to the requirement for the specification of forecast rainfall, which is a necessary input for physically-based models. To a lesser extent, improper estimation of the loss parameters is also a plausible reason, although the loss parameters can be reasonably estimated through calibration. Data driven models on the other hand have relatively less stringent requirements on input data, the water level (or discharge) at the location of interest being the minimum required. In practice, however; other inputs such as rainfall augmented with data from upstream stations are required to improve long term forecast results. Data driven models therefore represent viable alternatives to physically-based models in flow forecasting studies.

Keywords: Recursive Identification, Nonlinear System Identification, Other
Abstract: Rainfall/runoff models are used to forecast the flow of rivers and to prevent flood risks. In the north of France, the flood forecasting services employ models based on expert knowledge or grey-box models. However, the construction of such models rely on a relatively strong physical insight through for example the so-called evapotranspiration or humidity coefficient. In the objective of alleviating that reliance on physical knowledge, we take in this article a black-box modeling approach for the rainfall/runoff relation. A linear and a nonlinear model are estimated recursively online from real data collected on the Liane river (situated in the north of France). The obtained models are then used to predict potential occurrences of floods over a future horizon of 24 hours. Finally, performances of the proposed models are compared, and their advantages and drawbacks are highlighted.

Keywords: Multivariable System Identification, Subspace Methods, Other
Abstract: Flow routing models are an essential tool for water systems planning and management. In the last few years, advances in modelling capability, enhanced data availability, and increasing computational power have made the use of large, process-based, flow-routing models more common. However, the computational burden of such models still prevents their systematic application for operational purposes. This paper presents an alternative, data-driven approach to the identification of a flow-routing model both in one-step-ahead prediction and simulation mode, and demonstrates it by application to the Red River network, in Northern Vietnam. In particular, MIMO time-invariant models for the network are considered, their parameters are estimated using subspace model identification techniques and their performance is assessed with respect to the available data. Results show that the non-linear process of flow propagation is accurately reproduced both in prediction and simulation using very low order models.

Keywords: Other, Nonparametric Methods, Closed Loop Identification
Abstract: This paper demonstrates the feasibility of a new approach to system inversion and input signal estimation based on the exploitation of non-minimal state space feedback control system design methods that can be applied to non-minimum phase and unstable systems. The real and simulated examples demonstrate its practical utility and show that it has particular relevance in a hydrological systems context.

Keywords: Continuous Time System Estimation, Mechanical and Aerospace, Fault Detection and Diagnosis
Abstract: There are strong needs for health-monitoring of continuous, multi-dimensional structures as bridges or aircraft wings. Identification based on vibration analysis offer a promising option to fulfil such needs. The present contribution is dedicated to the use of Chebyshev orthogonal functions to expand the response of a system. Based on this expansion, a novel computation of the partial derivatives is presented}. Therefore the partial differential equations of motion can be written as algebraic equations. An exact expansion method is firstly presented. An {enhanced} differentiation method is then developed. For this aim, a case-study is provided: a cantilever beam which involves a fourth-order partial derivative with respect to space and a second-order partial derivative with respect to time of the displacement. {In this example, the identification error is reduced in comparison to classical methods. The proposed method has demonstrated it efficiency for identification of distributed parameter (material properties, structure dimensions). Therefore this identification method is well adapted for multi-dimensional structure monitoring.

Keywords: Continuous Time System Estimation, Hybrid and Distributed System Identification
Abstract: The paper discusses the emulation of large, distributed parameter, computer models by low order, continuous-time, transfer function models obtained using the SRIVC method of identification and estimation for continuous-time models. This yields a minimally parameterized, reduced order, 'nominal' emulation model that often reproduces the dynamic behavior of the large model to a remarkable degree. In full Dynamic Model Emulation (DEM), the objective is to emulate the high order model over a whole, user-defined range of parameter values, so that it can act as a surrogate for the high order model in applications that demand fast, repeated solution, as in Monte Carlo simulation and sensitivity analysis, or be used as a low order model in automatic control system design and adaptive forecasting applications. Most of the paper deals with the 'stand-alone' emulation of two high order, distributed parameter, computer models for the transport and dispersion of solutes in water systems.

Keywords: Nonlinear System Identification, Continuous Time System Estimation
Abstract: The continuous-time model identification problem for Hammerstein-Wiener systems with output measurement noise is studied. A simplified refined instrumental variable continuous-time (SRIVC) identification method is adopted. With the assumption of monotonic nonlinear function, the nonlinear model is iteratively estimated as an over-parameterized multiple-input single-output (MISO) linear time invariant model. Monte Carlo simulation analysis is presented to illustrate the effectiveness of the proposed method.

Keywords: Continuous Time System Estimation
Abstract: The fractional Ornstein-Uhlenbeck process is characterized by two parameters, the Hurst parameter of the fractional Brownian motion and the drift parameter. The problem of estimating the two parameters in a computationally efficient way from discrete-time data is considered in the paper. The covariance kernel of the Lamperti transformed fractional Ornstein-Uhlenbeck process in the stationary case is used in the analysis. It is shown least squares based estimation of the drift parameter does not give a consistent estimate in a general case as the number of data tends to infinity and the sampling interval tends to zero. A bias compensated estimate is therefore suggested, using an innovation variance based estimate of the Hurst parameter. The Cramér-Rao lower bound for the estimation of the parameters is evaluated and the properties of the suggested estimators are illustrated numerically.

Keywords: Nonlinear System Identification, Continuous Time System Estimation
Abstract: In this paper, it is demonstrated that an ultracapacitor exhibits a non-linear behaviour with respect to the operating voltage. For charge current sequences inducing low level voltage variations compared with the bias voltage accross its terminals, the analysis of the porous part the ultracapacitor impedance shows that the charge process can be modeled with the help of a continuous fractional model made of a classical integrator connected in parallel with a fractional integrator of order 0.5, whose action is blocked outside a limited spectral band. Furthermore, the investigated fractional model is able to provide an accurate estimation of the low frequency capacitance and high frequency internal resistance. In order to explicitly take into account the dependence of the system dynamics on the operating voltage, a fractional continuous LPV model is used and determined thanks to a local approach composed of two steps. First, the local LTI fractional model parameters are estimated thanks to an output-error technique at different bias voltages. Secondly, the parameter estimates dependence with respect to the operating voltage is obtained by using cubic spline interpolation. The resulting LPV fractional model is validated on an experimental ultracapacitor bench.

Keywords: Identification for Control, Bounded Error Identification, Error Quantification
Abstract: Inverse problems in science and engineering aim at estimating model parameters of a physical system using observations of the model’s response. Variational least square type approaches are typically adopted, solving the forward model, and then comparing the resulting modeled data with the actual measured data. The data mismatch is minimized and the process is iterated until the best match is achieved. However, data measurements are associated with uncertainties, and deterministic inverse algorithms hardly provide the associated error estimates for the model parameters. In this work, an interval-based iterative solution is presented to predict such errors, using adjoint-based optimization and the containment-stopping criterion.

Keywords: Bounded Error Identification
Abstract: We present a real-time robust positioning system based on interval analysis and constraint propagation that computes position domains, and that is able to detect and reject erroneous measurements. GPS pseudorange measurements are represented by intervals assumed to contain the true value with a given confidence level. A 3-D map of the drivable space is also available to constrain the vehicle location.

By the use of a breadth-first exploration strategy and of measurement consistency counters, the algorithm can be stopped at any time of the evaluation, and can instantly return the solution subpaving and the list of identified erroneous measurements. The method has been evaluated with real raw GPS data (i.e. pseudodistances on visible satellites) in a urban environment with a low-cost high-sensitivity GPS receiver providing faulty multipath measurements.

Keywords: Bounded Error Identification
Abstract: Interval analysis has been successfully applied to parameter and state estimation of noisy experimental data. However, obtaining efficient set estimators which are robust to outliers is still very difficult. This paper introduces new robust parameter and state estimation techniques in a bounded-error context in the presence of outliers. Several examples are provided concerning guaranteed parameter estimation of some chemical processes and the state tracking of aircrafts.

Keywords: Bounded Error Identification, Hybrid and Distributed System Identification, Other
Abstract: A wireless sensor network (WSN) consists of spatially distributed sensors connected via a wireless link. Sensors may be designed for pressure, temperature, sound, vibration, motion... This paper considers the problem of target tracking in a WSN. This problem is especially challenging in presence of measurements which are outliers. Two algorithms for target tracking robust to outliers are proposed. They only assume that the maximum number of outliers is known. Based on interval analysis, these algorithms perform a set-membership estimation using either SIVIA or a combinatorial technique. In both cases, sets of boxes guaranteed to contain the actual target location are provided.

Keywords: Bayesian Methods, Bounded Error Identification, Filtering and Smoothing
Abstract: This paper presents a MAP estimator for some vector x from its quantized and noisy linear measurements. The complexity of the optimal MAP estimator is intractable in general. Two suboptimal solutions have been proposed, one of which being iterative to be able to handle large-scale problems. Leveraging on techniques from interval analysis, it is possible to quickly eliminate solutions which are not consistent with the signal model, and the quantization noise.

These techniques have been applied to the estimation of the input signal of an OFB using noisy measurements of its quantized subbands. The experimental results show that when the channel is noisy, this approach performs better in terms of reconstruction SNR than classical least-squares reconstruction.

Keywords: Nonlinear System Identification
Abstract: This paper presents a new method for the identification of Wiener systems in the presence of output noise. The Wiener system identification problem is formulated as a convex Semidefinite Programming (SDP) problem by constraining a finite dimensional time dependency between signals. The main contribution of this paper is that the proposed method is robust to output noise and neither the Gaussian assumption of the input signal nor the invertibility of the static nonlinearity is necessary. The main assumption used in this paper is that static nonlinearity is monotonically non-decreasing. In the proposed identification method, the linear dynamical system is parametrized as a Finite Impulse Response (FIR) model and a nonparametric identification method is used to create the noise free output signal. Because both the intermediate signal and the noise free output signal are unknown, an over-parametrization technique is used. Once parameters are estimated, a Singular Value Decomposition (SVD) is used to separate the linear system parameters and the noise free output signal. The proposed identification method is applied to simulation data from a Wiener system. The effectiveness and accuracy of the proposed method are verified via numerical simulations.

Keywords: Recursive Identification, Nonlinear System Identification, Identifiability
Abstract: This note studies performance bounds of a Recursive Least Squares (RLS) algorithm and related second-order approaches. An important result is that their worst-case performance - or `regret' - grows only as O(dln N) in terms of the length N of the sequence at hand, and of the dimensionality d of the problem. Then, it is shown how to improve on this results for cases where d becomes large.The second main contribution of this paper is a derivation how this leads to an improved version of the RANKTRON algorithm for the recursive identification of a monotone Wiener system.

Keywords: Nonlinear System Identification, Basis Functions
Abstract: The class of Wiener-Schetzen models can describe a large variety of nonlinear systems. In this paper the dynamical part of these models is formulated in terms of orthonormal basis functions, while the nonlinearity is modeled through a multivariate polynomial. The parameters of the model are the coefficients of this polynomial. Generally this polynomial contains a relatively large number of significant terms, resulting in a large number of parameters. This paper considers a reduction of the significant parameters, by replacing one of the basis functions by the so-called best linear approximation of the system. It is shown that in this way the number of relevantly contributing terms in the multivariate polynomial is significantly reduced. Simulation results show a major reduction in the number of parameters, with only a minor increase in the rms error on the simulated output.

Keywords: Nonlinear System Identification, Grey Box Modelling
Abstract: This paper considers the identification of a rather general nonlinear time-invariant system, consisting of a Multiple-Input Multiple-Output (MIMO) linear dynamic part and one static nonlinear part. It is sometimes referred to as Linear Fractional Transformation (LFT) or Linear Fractional Representation (LFR). The structure will be called nonlinear LFR and includes many standard block-structured models, such as Wiener, Hammerstein, Wiener-Hammerstein and nonlinear feedback. The identification does assume neither the states, nor the internal signals over the static nonlinearity to be measured. The static nonlinearity (SNL) is assumed to be polynomial. After estimation of a nonlinear state-space model with certain structural properties, the SNL can be separated from the MIMO linear part. Next, the linear system is represented by a combination of four linear dynamic blocks, yielding extra insight. The method is illustrated via an experimental-data example.

Keywords: Particle Filtering/Monte Carlo Methods, Filtering and Smoothing, Nonlinear System Identification
Abstract: In this paper we discuss optimal filtering in general state-space models (SSMs) and present novel theoretical results on the long-term stability of bootstrap-type particle filters. More specifically, we establish that the asymptotic variance of the particle estimates is uniformly bounded in time. On the contrary to most previous results of this type, which in general presuppose that the state space of the hidden state process is compact (an assumption that is rarely satisfied in practice), our very mild assumptions are satisfied for a large class of SSMs with possibly non-compact state space. In addition, we derive similar stability results for the Lp error of the particle estimates. Importantly, our results hold for misspecified models, i.e., we do not at all assume that the data entering into the particle filter originate from the model governing the dynamics of the particles or not even from an SSM.

Keywords: Particle Filtering/Monte Carlo Methods, Bayesian Methods, Machine Learning and Data Mining
Abstract: We consider a semiparametric, i.e. a mixed parametric/nonparametric, model of a Wiener system. We use a state-space model for the linear dynamical system and a nonparametric Gaussian process (GP) model for the static nonlinearity. The GP model is a flexible model that can describe different types of nonlinearities while avoiding making strong assumptions such as monotonicity. We derive an inferential method based on recent advances in Monte Carlo statistical methods, known as Particle Markov Chain Monte Carlo (PMCMC). The idea underlying PMCMC is to use a particle filter (PF) to generate a sample state trajectory in a Markov chain Monte Carlo sampler. We use a recently proposed PMCMC sampler, denoted particle Gibbs with backward simulation, which has been shown to be efficient even when we use very few particles in the PF. The resulting method is used in a simulation study to identify two different Wiener systems with non-invertible nonlinearities.

Keywords: Nonlinear System Identification, Bayesian Methods
Abstract: This paper examines the problem of estimating the parameters describing system models of quite general nonlinear and multi-variable form. The approach is a computational one in which quantities that are intractable to evaluate exactly are approximated by sample averages from randomized algorithms. The main contribution is to illustrate the viability and utility of this approach by examining how high computational loads can be simply managed using commodity hardware. The proposed algorithms and solution architectures are profiled on concrete examples.

Keywords: Particle Filtering/Monte Carlo Methods, Nonlinear System Identification, Recursive Identification
Abstract: The particle filter (PF) has emerged as a powerful tool for solving nonlinear and/or non-Gaussian filtering problems. When some of the states enter the model linearly, this can be exploited by using particles only for the "nonlinear" states and employing conditional Kalman filters for the "linear" states; this leads to the Rao-Blackwellised particle filter (RBPF). However, it is well known that the PF fails when the state of the model contains some static parameter. This is true also for the RBPF, even if the static states are marginalised analytically by a Kalman filter. The reason is that the posterior density of the static states is computed conditioned on the nonlinear particle trajectories, which are bound to degenerate over time. To circumvent this problem, we propose a method for targeting the posterior parameter density, conditioned on just the current nonlinear state. This results in an RBPF-like method, capable of recursive identification of nonlinear dynamical models with affine parameter dependencies.

Keywords: Subspace Methods
Abstract: A constrained step-based realization algorithm is developed to produce linear, time-invariant, state-space system estimates. To match a priori knowledge of the system behavior, the eigenvalues of the estimate are required to be stable, real, and positive; the step response is required to have no undershoot or overshoot; and the steady-state gain is required to match a known value. The standard step-based realization method is augmented to become a convex optimization problem subject to a linear-matrix inequality that constrains eigenvalue location, and a subsequent convex optimization problem is developed to constrain time-domain behavior. Simulation results motivate the need for such constraints and are used for comparison with familiar alternative methods. Although the procedure is applied only to step-response data, it may be generalized to constrain eigenvalues to convex regions of the complex plain and is applicable to all subspace identification methods.

Keywords: Subspace Methods, Time Series
Abstract: This paper deals with estimation of the oscillatory components of a stationary process by subspace identification. Classical methods of harmonic retrieval work for the scalar case and do not seem to apply to vector-valued signals. We propose a subspace method which seems to deal quite satisfactorily with the problem even in the presence of additive colored noise.

Keywords: Subspace Methods, Error Quantification, Closed Loop Identification
Abstract: It is well-known that system identification is a valuable technique to obtain compact models for controller design and prediction. Subspace identification methods are of interest since they solely rely on tools from linear algebra and their ability to directly work with data generated by systems with multiple inputs and outputs. With respect to other methods (e.g. Prediction Error (PE)), subspace methods do not have straightforward asymptotic variance expressions. However, recently some papers appeared with asymptotic variance expressions for a certain class of identification algorithms that merge ideas from PE and subspace methods. In this paper we derive the asymptotic variance expression for the PBSIDopt algorithm and derive manageable expressions under some reasonable assumptions. We conclude the paper with two simulation examples where we show the strength of the proposed method.

Keywords: Subspace Methods, Continuous Time System Estimation, Multivariable System Identification
Abstract: The paper starts by presenting a new concept of differentiation similarity transformations for commensurate pseudo-states-space representations. It is proven that a pseudo-state-space representation with a commensurate differentiation order nu and a dimension of the transition matrix n can be similar to a pseudo-state-space representation with a commensurate order nu/k and a dimension of the transition matrix kn, where k is an integral number. A direct consequence of the aforementioned concept in fractional subspace-based identification methods for MIMO systems is that an overestimated pseudo-state-space representation has multiple minimums at commensurate differentiation orders over the integral number k. This result is especially visible when deterministic input/output signals are considered and less visible in the stochastic case due to overestimation.

Keywords: Bounded Error Identification
Abstract: In this paper we consider the problem of identifying a fixed-order FIR approximation of linear systems with unknown structure, assuming that both input and output measurements are subjected to quantization. In particular, a FIR model of given order which provides the best approximation of the input-output relationship is sought by minimizing the worst-case distance between the output of the true system and the modeled output, for all possible values of the input and output data consistent with their quantized measurements. First we show that the considered problem can be formulated in terms of robust optimization. Then, we present two different algorithms to compute the optimum of the formulated problem by means of linear programming techniques. The effectiveness of the proposed approach is illustrated by means of a simulation example.

Keywords: Recursive Identification
Abstract: We consider the identification of ARX systems which are observed via a binary sensor. Previous solutions typically assumed the complete knowledge of the noise distribution and that the inputs can be chosen by the user. Here, we only make mild assumptions on the noise and we make no assumptions that we can control the inputs. However, we assume that we can choose the threshold of the binary sensor or, equivalently, we can apply a dither signal to the system. We propose two recursive algorithms for this problem. The first one is based on an FIR approximation of the ARX system and requires post-processing. We prove that it provides a strongly consistent estimator. The other algorithm estimates the parameters directly, without post-processing, by simultaneously estimating the parameters and the outputs of the system. Numerical experiments which show that both algorithms work effectively are also presented.

Keywords: Bounded Error Identification
Abstract: Set membership system identification is based on the assumption that the uncertainties affecting the data or the model are unknown-but-bounded. When dealing with parametric models, the problem boils down to estimate the feasible parameter set, which is the set of all parameter values that are compatible with the available information. Unfortunately, feasible sets turn out to be nonconvex in several settings, including OE models with bounded output noise, ARX models with quantized measurements or errors-in-variables identification. In this paper, an algorithm is proposed for bounding nonconvex feasible sets, by exploiting the specific structure of the considered parametric models.

Keywords: Hybrid and Distributed System Identification, Nonlinear System Identification
Abstract: We present a novel identification framework that enables the use of first-order methods when estimating model parameters near a periodic orbit of a hybrid dynamical system. The proposed method reduces the space of initial conditions to a smooth manifold that contains the hybrid dynamics near the periodic orbit while maintaining the parametric dependence of the original hybrid model. First-order methods apply on this subsystem to minimize average prediction error, thus identifying parameters for the original hybrid system. We implement the technique and provide simulation results for a hybrid model relevant to terrestrial locomotion.

Keywords: Errors in Variables Identification, Other
Abstract: Multivariate polynomial system solving and polynomial optimization problems arise as central problems in many systems theory, identification and control settings. Traditionally, methods for solving polynomial equations have been developed in the area of algebraic geometry. Although a large body of literature is available, it is known as one of the most inaccessible fields of mathematics. In this paper we present a method for solving systems of polynomial equations employing numerical linear algebra and systems theory tools only, such as realization theory, SVD/QR, and eigenvalue computations. The task at hand is translated into the realm of linear algebra by separating coefficients and monomials into a coefficient matrix multiplied with a basis of monomials. Applying realization theory to the structure in the monomial basis allows to find all solutions of the system from eigenvalue computations. Solving a polynomial optimization problem is shown to be equivalent to an extremal eigenvalue problem. Relevant applications are found in identification and control, such as the global optimization of structured total least squares problems.

Keywords: Identification for Control
Abstract: The goal of parametric system identification is to provide estimates for parameters of a certain model structure based on given measurement data. This problem can always be presented as an optimization problem with an appropriate choice of cost and constraint functions. Apart from the simplest cases the resulting optimization problems are nonconvex with multiple local minima. Due to the existence of these, usually there are no guaranties that the model resulting from a given identification method is a global minimizer. This paper applies semidefinite programming (SDP) relaxation techniques to the optimization problem arising in time domain identification. From the solution of the defined sequence of SDPs a sequence of system models can be extracted that converges to the globally optimal system model. We give a short overview of the SDP relaxation technique for polynomial optimization problems (POP), then this technique is applied to the identification problem. The properties of the resulting POP are examined in detail. The solutions of the SDP sequence usually converge to the optimizer only in the limit. Model structures where finite convergence occurs and issues regarding detecting finite convergence are also considered.

Keywords: Filtering and Smoothing, Nonlinear System Identification, Maximum Likelihood Methods
Abstract: We propose two nonlinear Kalman smoothers that rely on Student’s t distributions. The t-Robust smoother, finds the maximum a posteriori likelihood (MAP) solution for Gaussian process noise and Student’s t observation noise, and is extremely robust against outliers, outperforming the recently proposed l1-Laplace smoother in extreme situations (e.g. 50% or more outliers). The second estimator, which we call the t-Trend smoother, is able to follow sudden changes in the process model, and is derived as a MAP solver for a model with Student’s t-process noise and Gaussian observation noise. We design specialized methods to solve both problems which exploit the special structure of the Student’s t-distribution, and provide a convergence theory. Both smoothers can be implemented with only minor modifications to an existing L2 smoother implementation. Numerical results for linear and nonlinear models illustrating both robust and fast tracking applications are presented.

Keywords: Filtering and Smoothing, Continuous Time System Estimation, Bayesian Methods
Abstract: This paper is concerned with application of cubature integration methods to Kalman filtering of discretely observed non-linear stochastic continuous-time systems. We compare two recently proposed variants of the continuous-discrete cubature Kalman filter (CD-CKF), which differ in the order how the discretization and the Gaussian approximation are done. Aside with theoretical analysis we test the performance of the different variants in a simulated application. The results indicate that the relative advantages of the approaches are application specific and neither one is unconditionally better than the other.

Keywords: Identification for Control, Machine Learning and Data Mining, Process Control
Abstract: A system identification method using predictive models based on the imitation of analyst’s associative thinking is developed. Identification algorithms using associative search for description of the knowledge about control plant are presented. Wavelet analysis is applied for time-varying objects identification. The paper has been supported by a grant of the Russian Foundation for Basic Researches (RFBR): project 12-07-00577-a.

Keywords: Identification for Control, Nonlinear System Identification, Other
Abstract: This article considers the identification of state-dependent parameter (SDP) models for the hydraulically actuated dual-manipulators of a mobile robot used for nuclear decommissioning tasks. A unified framework for calibration, data collection and system identification is developed, and utilized to investigate potential state-dependencies. The latter are associated with nonlinear system dynamics and can cause irregular joint movements when the device is controlled using linear control algorithms. The analysis suggests that a univariate SDP model is suitable for control design. The model has a state-dependent gain, characterized directly from experimental data using a numerically optimized polynomial function of the delayed input variable. In order to demonstrate the practical utility of the SDP model, closed-loop results using a novel non-minimal regulator for joint control are briefly considered.

Keywords: Identification for Control, Closed Loop Identification
Abstract: This paper takes a class of first-order systems as an example to study the adaptive tracking control via binary-valued observations with fixed threshold. Using the statistical property of the system noise, a projection algorithm is proposed for parameter estimation. Then, the adaptive control law is designed via the certainty equivalence principle. By use of the conditional expectations of the innovation and output prediction with respect to the estimates, the closed-loop system and adaptive control law are shown to be stable and asymptotically optimal. Meanwhile, the parameter estimate is proved to be both almost surely and mean square convergent, and the convergent rate of the estimation error is also obtained. A numerical example is given to demonstrate the efficiency of the adaptive control law.

Keywords: Identification for Control, Closed Loop Identification, Multivariable System Identification
Abstract: In this paper, the discrete-time exponential forgetting factor observer for state affine systems is specialized for the joint estimation of state and parameters, leading to an adaptive observer in which the decay rates of the estimation errors for the two distinct objects to be estimated can be disjoined. Experimental results for the adaptive observation of rotor time constant and stator resistance of an induction motor are presented as example of application.

Keywords: Multivariable System Identification, Grey Box Modelling, Mechanical and Aerospace
Abstract: In this paper, a methodology is investigated for the determination of a dynamical model of robotic manipulators. Rather than building a model either from the law of Physics or from experimental data independently, a combination of an analytical and an experimental approach is performed in order to identify a linear parameter-varying (LPV) model of the system. A local approach dedicated to LPV models is developed. As a sample, the case of a 2-DOF flexible manipulator is addressed. The identification procedure is evaluated from simulated data. This study shows that an LPV model with polynomial variation of the state equation matrices with respect to one scheduling parameter provides good results in terms of output fit.

Keywords: Nonlinear System Identification, Identification for Control, Mechanical and Aerospace
Abstract: Braking is recognized to be one of the most critical and sensitive maneuvers by most motorbikers and race-engineers. Many recent studies have been devoted to model-based design of electronic control systems to enhance the driver’s safety. In this work, a linear parameter-varying approach for braking control is proposed, based on the observation that in motorcycles the load transfer strongly affects the vehicle dynamics. Since modeling a real-world system might be very time-consuming, a direct data-driven approach is employed to tune the controller parameters, without need of a mathematical description of the system. The strategy is implemented on a full-fledged multibody simulator, and results are compared to a standard model-based strategy.

Keywords: Grey Box Modelling, Frequency Domain Identification, Subspace Methods
Abstract: This paper studies grey-box modeling of a rotary type pendulum system with a position-variable load, which can be modeled by Parameter-Dependent Linear-Time-Invariant (PD-LTI) systems. The state of the system is estimated from physical laws with black-box models, and the coefficients of the PD-LTI system are computed from LMI (Linear Matrix Inequalities) optimization. Two white-box models are developed and the resulting grey-box models are verified by experiments.

Keywords: Nonlinear System Identification, Process Control
Abstract: This work studies the identification of LPV (linear parameter varying) models with two scheduling variables in order to model complex industrial processes. The LPV model is parameterized as blended linear models, which is also called multi-model approach. Several weighting functions, linear, polynomial and Gaussian functions, are used and compared. The usefulness of the method is tested using a high purity distillation column model in a case study. The case study also shows that a good fit of identification data is not enough to verify model quality and can even be misleading in nonlinear process identification; other measures related to process knowledge should be used in model validation.