View source for Linear time-invariant system

{{Short description|Mathematical model which is both linear and time-invariant}}
{{More footnotes|date=April 2009}}

[[File:Superposition principle and time invariance block diagram for a SISO system.png|320px|thumb|[[Block diagram]] illustrating the [[superposition principle]] and time invariance for a deterministic continuous-time single-input single-output system. The system satisfies the [[superposition principle]] and is time-invariant if and only if {{math|1=''y''{{sub|3}}(''t'') = ''a''{{sub|1}}''y''{{sub|1}}(''t'' – ''t''{{sub|0}}) + ''a''{{sub|2}}''y''{{sub|2}}(''t'' – ''t''{{sub|0}})}} for all time {{mvar|t}}, for all real constants {{math|''a''{{sub|1}}, ''a''{{sub|2}}, ''t''{{sub|0}}}} and for all inputs {{math|''x''{{sub|1}}(''t''), ''x''{{sub|2}}(''t'')}}.<ref name="Bessai_2005">{{cite book | title = MIMO Signals and Systems | first = Horst J. | last = Bessai | publisher = Springer | year = 2005 | pages = 27–28 | isbn = 0-387-23488-8}}</ref> Click image to expand it.]]

In [[system analysis]], among other fields of study, a '''linear time-invariant''' ('''LTI''') '''system''' is a [[system]] that produces an output signal from any input signal subject to the constraints of [[Linear system#Definition|linearity]] and [[Time-invariant system|time-invariance]]; these terms are briefly defined [[#Overview|below]]. These properties apply (exactly or approximately) to many important physical systems, in which case the response {{math|''y''(''t'')}} of the system to an arbitrary input {{math|''x''(''t'')}} can be found directly using [[convolution]]: {{math|1=''y''(''t'') = (''x'' ∗ ''h'')(''t'')}} where {{math|''h''(''t'')}} is called the system's [[impulse response]] and ∗ represents convolution (not to be confused with multiplication). What's more, there are systematic methods for solving any such system (determining {{math|''h''(''t'')}}), whereas systems not meeting both properties are generally more difficult (or impossible) to solve analytically. A good example of an LTI system is any [[electrical circuit]] consisting of [[resistor]]s, [[capacitor]]s, [[inductor]]s and [[linear amplifier]]s.<ref>Hespanha 2009, p. 78.</ref>

Linear time-invariant system theory is also used in [[image processing]], where the systems have spatial dimensions instead of, or in addition to, a temporal dimension. These systems may be referred to as ''linear translation-invariant'' to give the terminology the most general reach. In the case of generic [[discrete-time]] (i.e., [[sample (signal)|sampled]]) systems, ''linear shift-invariant'' is the corresponding term. LTI system theory is an area of [[applied mathematics]] which has direct applications in [[Network analysis (electrical circuits)|electrical circuit analysis and design]], [[signal processing]] and [[filter design]], [[control theory]], [[mechanical engineering]], [[image processing]], the design of [[measuring instrument]]s of many sorts, [[NMR spectroscopy]]{{Citation needed|date=September 2020}}, and many other technical areas where systems of [[ordinary differential equation]]s present themselves.

==Overview==
The defining properties of any LTI system are ''linearity'' and ''time invariance''.
* ''Linearity'' means that the relationship between the input <math>x(t)</math> and the output <math>y(t)</math>, both being regarded as functions, is a linear mapping: If <math>a</math> is a constant then the system output to <math>ax(t)</math> is <math>ay(t)</math>; if <math>x'(t)</math> is a further input with system output <math>y'(t)</math> then the output of the system to <math>x(t)+x'(t)</math> is <math>y(t)+y'(t)</math>, this applying for all choices of <math>a</math>'','' ''<math>x(t)</math>'', <math>x'(t)</math>. The latter condition is often referred to as the [[superposition principle]].
* ''Time invariance'' means that whether we apply an input to the system now or ''T'' seconds from now, the output will be identical except for a time delay of ''T'' seconds. That is, if the output due to input <math>x(t)</math> is <math>y(t)</math>, then the output due to input <math>x(t-T)</math> is <math>y(t-T)</math>. Hence, the system is time invariant because the output does not depend on the particular time the input is applied.

The fundamental result in LTI system theory is that any LTI system can be characterized entirely by a single function called the system's [[impulse response]].  The output of the system <math>y(t)</math> is simply the [[convolution]] of the input to the system <math>x(t)</math> with the system's impulse response <math>h(t)</math>. This is called a [[continuous time]] system. Similarly, a discrete-time linear time-invariant (or, more generally, "shift-invariant") system is defined as one operating in [[discrete time]]: <math>y_{i} = x_{i} * h_{i}</math>  where ''y'', ''x'', and ''h'' are [[sequences]] and the convolution, in discrete time, uses a discrete summation rather than an integral.

[[File:LTI.png|thumb|Relationship between the '''time domain''' and the '''frequency domain'''|right|320px]]

LTI systems can also be characterized in the ''[[frequency domain]]'' by the system's [[transfer function]], which is the [[Laplace transform]] of the system's impulse response (or [[Z transform]] in the case of discrete-time systems).  As a result of the properties of these transforms, the output of the system in the frequency domain is the product of the transfer function and the transform of the input. In other words, convolution in the time domain is equivalent to multiplication in the frequency domain.

For all LTI systems, the [[eigenfunction]]s, and the basis functions of the transforms, are [[complex number|complex]] [[exponential function|exponentials]].  This is, if the input to a system is the complex waveform <math>A_s e^{st}</math> for some complex amplitude <math>A_s</math> and complex frequency <math>s</math>, the output will be some complex constant times the input, say <math>B_s e^{st}</math> for some new complex amplitude <math>B_s</math>.  The ratio <math>B_s/A_s</math> is the transfer function at frequency <math>s</math>.

Since [[sine wave|sinusoids]] are a sum of complex exponentials with complex-conjugate frequencies, if the input to the system is a sinusoid, then the output of the system will also be a sinusoid, perhaps with a different [[amplitude]] and a different [[phase (waves)|phase]], but always with the same frequency upon reaching steady-state. LTI systems cannot produce frequency components that are not in the input.

LTI system theory is good at describing many important systems.  Most LTI systems are considered "easy" to analyze, at least compared to the time-varying and/or [[nonlinear]] case.  Any system that can be modeled as a linear [[differential equation]] with constant coefficients is an LTI system.  Examples of such systems are [[electrical network|electrical circuits]] made up of [[resistor]]s, [[inductor]]s, and [[capacitor]]s (RLC circuits).  Ideal spring–mass–damper systems are also LTI systems, and are mathematically equivalent to RLC circuits.

Most LTI system concepts are similar between the continuous-time and discrete-time (linear shift-invariant) cases.  In image processing, the time variable is replaced with two space variables, and the notion of time invariance is replaced by two-dimensional shift invariance.  When analyzing [[filter bank]]s and [[MIMO (systems theory)|MIMO]] systems, it is often useful to consider [[matrix (mathematics)|vectors]] of signals.

A linear system that is not time-invariant can be solved using other approaches such as the [[Green's function|Green function]] method.

== Continuous-time systems ==

===Impulse response and convolution===
The behavior of a linear, continuous-time, time-invariant system with input signal ''x''(''t'') and output signal ''y''(''t'') is described by the convolution integral:<ref>Crutchfield, p. 1. ''Welcome!''</ref>
:{|
| <math>y(t) = (x * h)(t)</math>
| <math>\mathrel{\stackrel{\mathrm{def}}{=}} \int\limits_{-\infty}^{\infty} x(t - \tau)\cdot h(\tau) \, \mathrm{d}\tau</math>
|-
|
| <math>= \int\limits_{-\infty}^\infty x(\tau)\cdot h(t - \tau) \,\mathrm{d}\tau,</math> {{spaces|5}} (using [[Convolution#Commutativity|commutativity]])
|}

where <math display="inline"> h(t)</math> is the system's response to an [[Dirac delta function|impulse]]: <math display="inline">x(\tau) = \delta(\tau)</math>. <math display="inline"> y(t) </math> is therefore proportional to a weighted average of the input function <math display="inline">x(\tau)</math>. The weighting function is <math display="inline"> h(-\tau)</math>, simply shifted by amount <math display="inline"> t</math>. As <math display="inline"> t</math> changes, the weighting function emphasizes different parts of the input function. When <math display="inline"> h(\tau)</math> is zero for all negative <math display="inline"> \tau</math>, <math display="inline"> y(t)</math> depends only on values of <math display="inline"> x</math> prior to time <math display="inline"> t</math>, and the system is said to be [[Causal system|causal]].

To understand why the convolution produces the output of an LTI system, let the notation <math display="inline"> \{x(u-\tau);\ u\}</math> represent the function <math display="inline"> x(u-\tau)</math> with variable <math display="inline"> u</math> and constant <math display="inline"> \tau</math>. And let the shorter notation <math display="inline"> \{x\}</math> represent <math display="inline"> \{x(u);\ u\}</math>. Then a continuous-time system transforms an input function, <math display="inline"> \{x\},</math> into an output function, <math display="inline">\{y\}</math>. And in general, every value of the output can depend on every value of the input. This concept is represented by:
<math display="block">y(t) \mathrel{\stackrel{\text{def}}{=}} O_t\{x\},</math>
where <math display="inline"> O_t</math> is the transformation operator for time <math display="inline"> t</math>. In a typical system, <math display="inline"> y(t)</math> depends most heavily on the values of <math display="inline"> x</math> that occurred near time <math display="inline"> t</math>. Unless the transform itself changes with <math display="inline"> t</math>, the output function is just constant, and the system is uninteresting.

For a linear system, <math display="inline"> O</math> must satisfy {{EquationNote|Eq.1}}:

{{NumBlk|:|<math>
O_t\left\{\int\limits_{-\infty}^\infty c_{\tau}\ x_{\tau}(u) \, \mathrm{d}\tau ;\ u\right\} = \int\limits_{-\infty}^\infty c_\tau\ \underbrace{y_\tau(t)}_{O_t\{x_\tau\}} \, \mathrm{d}\tau.
</math>|{{EquationRef|Eq.2}}}}

And the time-invariance requirement is:

{{NumBlk|:|<math>
\begin{align}
  O_t\{x(u - \tau);\ u\} &\mathrel{\stackrel{\quad}{=}} y(t - \tau)\\
                         &\mathrel{\stackrel{\text{def}}{=}} O_{t-\tau}\{x\}.\,
\end{align}
</math>
| {{EquationRef|Eq.3}}
}}

In this notation, we can write the '''impulse response''' as <math display="inline"> h(t) \mathrel{\stackrel{\text{def}}{=}} O_t\{\delta(u);\ u\}.</math>

Similarly:
:{|
| <math>h(t - \tau)</math>
| <math>\mathrel{\stackrel{\text{def}}{=}} O_{t-\tau}\{\delta(u);\ u\}</math>
|-
|
| <math>= O_t\{\delta(u - \tau);\ u\}.</math> {{spaces|5}} (using {{EquationNote|Eq.3}})
|}

Substituting this result into the convolution integral:
<math display="block">
\begin{align}
  (x * h)(t) &= \int_{-\infty}^\infty x(\tau)\cdot h(t - \tau) \,\mathrm{d}\tau \\[4pt]
             &= \int_{-\infty}^\infty x(\tau)\cdot O_t\{\delta(u-\tau);\ u\} \, \mathrm{d}\tau,\,
\end{align}
</math>

which has the form of the right side of {{EquationNote|Eq.2}} for the case <math display="inline"> c_\tau = x(\tau)</math> and <math display="inline"> x_\tau(u) = \delta(u-\tau).</math>

{{EquationNote|Eq.2}} then allows this continuation:
<math display="block">
\begin{align}
  (x * h)(t) &= O_t\left\{\int_{-\infty}^\infty x(\tau)\cdot \delta(u-\tau) \, \mathrm{d}\tau;\ u \right\}\\[4pt]
             &= O_t\left\{x(u);\ u \right\}\\
             &\mathrel{\stackrel{\text{def}}{=}} y(t).\,
\end{align}
</math>

In summary, the input function, <math display="inline"> \{x\}</math>, can be represented by a continuum of time-shifted impulse functions, combined "linearly", as shown at {{EquationRef|Eq.1}}.  The system's linearity property allows the system's response to be represented by the corresponding continuum of impulse <u>responses</u>, combined in the same way. And the time-invariance property allows that combination to be represented by the convolution integral.

The mathematical operations above have a simple graphical simulation.<ref>Crutchfield, p. 1. ''Exercises''</ref>

=== Exponentials as eigenfunctions ===
An [[eigenfunction]] is a function for which the output of the operator is a scaled version of the same function. That is,
<math display="block">\mathcal{H}f = \lambda f,</math>
where ''f'' is the eigenfunction and <math>\lambda</math> is the [[eigenvalue]], a constant.

The [[exponential function]]s <math>A e^{s t}</math>, where <math>A, s \in \mathbb{C}</math>, are [[eigenfunction]]s of a [[linear]], [[time-invariant]] operator. A simple proof illustrates this concept. Suppose the input is <math>x(t) = A e^{s t}</math>. The output of the system with impulse response <math>h(t)</math> is then
<math display="block">\int_{-\infty}^\infty h(t - \tau) A e^{s \tau}\, \mathrm{d} \tau</math>
which, by the commutative property of [[convolution]], is equivalent to
<math display="block">\begin{align}
    \overbrace{\int_{-\infty}^\infty h(\tau) \, A e^{s (t - \tau)} \, \mathrm{d} \tau}^{\mathcal{H} f}
  &= \int_{-\infty}^\infty h(\tau) \, A e^{s t} e^{-s \tau} \, \mathrm{d} \tau \\[4pt]
  &= A e^{s t} \int_{-\infty}^{\infty} h(\tau) \, e^{-s \tau} \, \mathrm{d} \tau \\[4pt]
  &= \overbrace{\underbrace{A e^{s t}}_{\text{Input}}}^{f} \overbrace{\underbrace{H(s)}_{\text{Scalar}}}^{\lambda}, \\
\end{align}</math>

where the scalar
<math display="block">H(s) \mathrel{\stackrel{\text{def}}{=}} \int_{-\infty}^\infty h(t) e^{-s t} \, \mathrm{d} t</math>
is dependent only on the parameter ''s''.

So the system's response is a scaled version of the input. In particular, for any <math>A, s \in \mathbb{C}</math>, the system output is the product of the input <math>A e^{st}</math> and the constant <math>H(s)</math>. Hence, <math>A e^{s t}</math> is an [[eigenfunction]] of an LTI system, and the corresponding [[eigenvalue]] is <math>H(s)</math>.

==== Direct proof ====
It is also possible to directly derive complex exponentials as eigenfunctions of LTI systems.

Let's set <math>v(t) = e^{i \omega t}</math> some complex exponential and <math>v_a(t) = e^{i \omega (t+a)}</math> a time-shifted version of it.

<math>H[v_a](t) = e^{i\omega a} H[v](t)</math> by linearity with respect to the constant <math>e^{i \omega a}</math>.

<math>H[v_a](t) = H[v](t+a)</math> by time invariance of <math>H</math>.

So <math>H[v](t+a) = e^{i \omega a} H[v](t)</math>. Setting <math>t = 0</math> and renaming we get:
<math display="block">H[v](\tau) = e^{i\omega \tau} H[v](0)</math>
i.e. that a complex exponential <math>e^{i \omega \tau}</math> as input will give a complex exponential of same frequency as output.

=== Fourier and Laplace transforms ===
The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems.  The one-sided [[Laplace transform]]
<math display="block">H(s) \mathrel{\stackrel{\text{def}}{=}} \mathcal{L}\{h(t)\} \mathrel{\stackrel{\text{def}}{=}} \int_0^\infty h(t) e^{-s t} \, \mathrm{d} t</math>
is exactly the way to get the eigenvalues from the impulse response.  Of particular interest are pure sinusoids (i.e., exponential functions of the form <math>e^{j \omega t}</math> where <math>\omega \in \mathbb{R}</math> and <math>j \mathrel{\stackrel{\text{def}}{=}} \sqrt{-1}</math>). The [[Fourier transform]] <math>H(j \omega) = \mathcal{F}\{h(t)\}</math> gives the eigenvalues for pure complex sinusoids. Both of <math>H(s)</math> and <math>H(j\omega)</math> are called the ''system function'', ''system response'', or ''transfer function''.

The Laplace transform is usually used in the context of one-sided signals, i.e. signals that are zero for all values of ''t'' less than some value. Usually, this "start time" is set to zero, for convenience and without loss of generality, with the transform integral being taken from zero to infinity (the transform shown above with lower limit of integration of negative infinity is formally known as the [[bilateral Laplace transform]]).

The Fourier transform is used for analyzing systems that process signals that are infinite in extent, such as modulated sinusoids, even though it cannot be directly applied to input and output signals that are not [[square integrable]].  The Laplace transform actually works directly for these signals if they are zero before a start time, even if they are not square integrable, for stable systems.  The Fourier transform is often applied to spectra of infinite signals via the  [[Wiener–Khinchin theorem]] even when Fourier transforms of the signals do not exist.

Due to the convolution property of both of these transforms, the convolution that gives the output of the system can be transformed to a multiplication in the transform domain, given signals for which the transforms exist
<math display="block">y(t) = (h*x)(t) \mathrel{\stackrel{\text{def}}{=}} \int_{-\infty}^\infty h(t - \tau) x(\tau) \, \mathrm{d} \tau \mathrel{\stackrel{\text{def}}{=}} \mathcal{L}^{-1}\{H(s)X(s)\}.</math>

One can use the system response directly to determine how any particular frequency component is handled by a system with that Laplace transform. If we evaluate the system response (Laplace transform of the impulse response) at complex frequency {{nowrap|''s'' {{=}} ''jω''}}, where {{nowrap|''ω'' {{=}} 2''πf''}}, we obtain |''H''(''s'')| which is the system gain for frequency ''f''. The relative phase shift between the output and input for that frequency component is likewise given by arg(''H''(''s'')).

=== Examples ===
{{bulleted list
| A simple example of an LTI operator is the [[derivative]].
* <math> \frac{\mathrm{d}}{\mathrm{d}t} \left( c_1 x_1(t) + c_2 x_2(t) \right) = c_1 x'_1(t) + c_2 x'_2(t) </math> &nbsp; (i.e., it is linear)
* <math> \frac{\mathrm{d}}{\mathrm{d}t} x(t-\tau) = x'(t-\tau) </math> &nbsp; (i.e., it is time invariant)
When the Laplace transform of the derivative is taken, it transforms to a simple multiplication by the Laplace variable ''s''.
<math display="block"> \mathcal{L}\left\{\frac{\mathrm{d}}{\mathrm{d}t}x(t)\right\} = s X(s) </math>
That the derivative has such a simple Laplace transform partly explains the utility of the transform.
| Another simple LTI operator is an averaging operator
<math display="block"> \mathcal{A}\left\{x(t)\right\} \mathrel{\stackrel{\text{def}}{=}} \int_{t-a}^{t+a} x(\lambda) \, \mathrm{d} \lambda. </math>

By the linearity of integration,
<math display="block">\begin{align}
  \mathcal{A} \{c_1 x_1(t) + c_2 x_2(t)\}
    &= \int_{t-a}^{t+a} ( c_1 x_1(\lambda) + c_2 x_2(\lambda)) \, \mathrm{d} \lambda\\
    &= c_1 \int_{t-a}^{t+a} x_1(\lambda) \, \mathrm{d} \lambda + c_2 \int_{t-a}^{t+a} x_2(\lambda) \, \mathrm{d} \lambda\\
    &= c_1 \mathcal{A}\{x_1(t)\} + c_2 \mathcal{A} \{x_2(t) \},
\end{align}</math>

it is linear. Additionally, because
<math display="block">\begin{align}
  \mathcal{A}\left\{x(t-\tau)\right\}
    &= \int_{t-a}^{t+a} x(\lambda-\tau) \, \mathrm{d} \lambda\\
    &= \int_{(t-\tau)-a}^{(t-\tau)+a} x(\xi) \, \mathrm{d} \xi\\
    &= \mathcal{A}\{x\}(t-\tau),
\end{align}</math>

it is time invariant. In fact, <math>\mathcal{A}</math> can be written as a convolution with the [[boxcar function]] <math>\Pi(t)</math>. That is,
<math display="block">\mathcal{A}\left\{x(t)\right\} = \int_{-\infty}^\infty \Pi\left(\frac{\lambda-t}{2a}\right) x(\lambda) \, \mathrm{d} \lambda,</math>

where the boxcar function
<math display="block">\Pi(t) \mathrel{\stackrel{\text{def}}{=}} \begin{cases}
    1 &\text{if } |t| < \frac{1}{2},\\
    0 &\text{if } |t| > \frac{1}{2}.
  \end{cases}</math>
}}

=== Important system properties ===
Some of the most important properties of a system are causality and stability.  Causality is a necessity for a physical system whose independent variable is time, however this restriction is not present in other cases such as image processing.

==== Causality ====
{{Main|Causal system}}
<!--the causal system article needs work-->
A system is causal if the output depends only on present and past, but not future inputs.  A necessary and sufficient condition for causality is
<math display="block">h(t) = 0 \quad \forall t < 0,</math>

where <math>h(t)</math> is the impulse response.  It is not possible in general to determine causality from the [[two-sided Laplace transform]]. However when working in the time domain one normally uses the [[Laplace transform|one-sided Laplace transform]] which requires causality.

==== Stability ====
{{Main|BIBO stability}}
A system is '''bounded-input, bounded-output stable''' (BIBO stable) if, for every bounded input, the output is finite.  Mathematically, if every input satisfying
<math display="block">\ \|x(t)\|_{\infty} < \infty</math>

leads to an output satisfying
<math display="block">\ \|y(t)\|_{\infty} < \infty</math>

(that is, a finite [[Infinity norm|maximum absolute value]] of <math>x(t)</math> implies a finite maximum absolute value of <math>y(t)</math>), then the system is stable.  A necessary and sufficient condition is that <math>h(t)</math>, the impulse response, is in [[Lp space|L<sup>1</sup>]] (has a finite L<sup>1</sup> norm):
<math display="block">\|h(t)\|_1 = \int_{-\infty}^\infty |h(t)| \, \mathrm{d}t < \infty.</math>

In the frequency domain, the [[region of convergence]] must contain the imaginary axis <math>s = j\omega</math>.

As an example, the ideal [[low-pass filter]] with impulse response equal to a [[sinc function]] is not BIBO stable, because the sinc function does not have a finite L<sup>1</sup> norm.  Thus, for some bounded input, the output of the ideal low-pass filter is unbounded.  In particular, if the input is zero for <math>t < 0</math> and equal to a sinusoid at the [[cut-off frequency]] for <math>t > 0</math>, then the output will be unbounded for all times other than the zero crossings.{{dubious|date=September 2020}}


=== Deriving the Solution of Linear Time-Invariant Differential Equations ===
Given is an explicit linear system of differential equations in the form:
:<math>
\begin{align}
 \frac{dx(t)}{dt}=A\,x(t)+b\,u(t), \,x(t=0)=x_{0}
\end{align}
</math>
with the state vector <math> x(t)\in \mathbb{R}^{n} </math>, the system matrix <math>A\in\mathbb{R}^{n\times n} </math>, the input <math> u(t)\in \mathbb{R} </math>, the input vector <math> b\in \mathbb{R}^{n} </math> and the initial condition <math> x_{0} \in \mathbb{R}^{n}</math>. The solution consists of a homogeneous and a particular part.

==== Homogeneous solution ====
The homogeneous differential equation is obtained by setting the input equal to zero.

:<math>\begin{align}
 \frac{dx(t)}{dt}=A\,x(t),\, x(t=0)=x_{0}

\end{align}
</math>
This solution can now be described using a [[Taylor series]] representation:
:<math>
\begin{align}
 x(t)=\phi(t)x_{0}=(E+\phi_{1}t+\phi_{2}t^{2}+...+\phi_{n}t^{n}+...)x_0

\end{align}
</math>
where <math> E </math> is the unit matrix.
Substituting this solution into the above equation, one obtains:
:<math>
\begin{align}

 \frac{d}{dt}(\phi(t)x_0)&=A\,\phi(t)\,x_{0}\\
 (\phi_{1}+2\,\phi_{2}\,t+...+n\,\phi_{n}\,t^{n-1}+...)x_0&=(A+A\phi_{1}t+A\phi_{2}t^{2}+...+A\phi_{n}t^{n}+...)\,x_0

\end{align}
</math>
Now the unknown matrices <math> \phi_{n} </math> can be determined by comparing coefficients:
:<math>
\begin{align}

  \phi_{1}  &=  A \\
  \phi_{2}&=\frac{1}{2}A\,\phi_{1}=\frac{1}{2!}A^{^2} \\
  \phi_{3}&=\frac{1}{3}A\,\phi_{2}=\frac{1}{3!}A^{3} \\
  &... \\
  \phi_{n}&=\frac{1}{n!}A^{n}.

\end{align}
</math>
The following notation is commonly used for the fundamental matrix <math> \phi_{n} </math>:
:<math>
\begin{align}
\phi(t)=e^{At}=E+At+\frac{1}{2!}A^{2}t^{2}+\frac{1}{3!}A^{3}t^{3}+...+\frac{1}{n!}A^{n}t^{n}+...
\end{align}
</math>

==== Particular solution ====
Assuming <math> u(t)\neq0 </math> and <math> x_{0}=0 </math>, follows:
:<math>
\begin{align}
\frac{d}{dt}x(t)=A\,x(t)+b\,u(t)

\end{align}
</math>
The particular solution is obtained in the form:
:<math>
\begin{align}
x_{p}(t)=\phi(t)\xi(t)=e^{At}\xi(t),

\end{align}
</math>
where <math> \xi(t) </math> is an unknown function vector with <math> \xi(0)=0 </math>. From the above two equations follows:
:<math>
\begin{align}

\frac{d}{dt}x_{p}(t)&=A\,x_{p}(t)+b\,u(t)\\
\xi(t)\frac{d}{dt}\phi(t)+\phi(t)\frac{d}{dt}\xi(t)&=A\,x_{p}(t)+b\,u(t)\\
A\,\phi(t)\xi(t)+\phi(t)\frac{d}{dt}\xi(t)&=A\,x_{p}(t)+b\,u(t)\\
A\,x_{p}(t)+\phi(t)\frac{d}{dt}\xi(t)&=A\,x_{p}(t)+b\,u(t)\\

\end{align}
</math>
Thus <math> \xi(t) </math> can be determined:
:<math>
\begin{align}
\frac{d}{dt}\xi(t)=\phi^{-1}(t)bu(t)\\
\end{align}
</math>
One obtains by integration utilizing the properties of the fundamental matrix:
:<math>
\begin{align}

\phi(t)\xi(t)&=\phi(t)\int_{0}^{t}\phi^{-1}(\tau)bu(\tau)d\tau\\
x_{p}(t)&=\int_{0}^{t}\phi(t-\tau)bu(\tau)d\tau\\
x_{p}(t)&=\int_{0}^{t}e^{A(t-\tau)}bu(\tau)d\tau\\

\end{align}
</math>
Thus, we finally obtain the solution of a linear time-invariant differential equation:
:<math>
\begin{align}
x(t)=e^{At}x_{0} + \int_{0}^{t}e^{A(t-\tau)}bu(\tau)d\tau\\
\end{align}
</math>

== Discrete-time systems ==
Almost everything in continuous-time systems has a counterpart in discrete-time systems.<!-- this section may be very redundant.  don't remove this redundancy because these should probably be separate articles. -->

=== Discrete-time systems from continuous-time systems ===
In many contexts, a discrete time (DT) system is really part of a larger continuous time (CT) system.  For example, a digital recording system takes an analog sound, digitizes it, possibly processes the digital signals, and plays back an analog sound for people to listen to.

In practical systems, DT signals obtained are usually uniformly sampled versions of CT signals.  If <math>x(t)</math> is a CT signal, then the [[Sample and hold|sampling circuit]] used before an [[analog-to-digital converter]] will transform it to a DT signal:
<math display="block">x_n \mathrel{\stackrel{\text{def}}{=}} x(nT) \qquad \forall \, n \in \mathbb{Z},</math>
where ''T'' is the [[sampling frequency|sampling period]]. Before sampling, the input signal is normally run through a so-called [[anti-aliasing filter|Nyquist filter]] which removes frequencies above the "folding frequency" 1/(2T); this guarantees that no information in the filtered signal will be lost. Without filtering, any frequency component ''above'' the folding frequency (or [[Nyquist frequency]]) is [[Aliasing|aliased]] to a different frequency (thus distorting the original signal), since a DT signal can only support frequency components lower than the folding frequency.

=== Impulse response and convolution ===

Let <math>\{x[m - k];\ m\}</math> represent the sequence <math>\{x[m - k];\text{ for all integer values of } m\}.</math>

And let the shorter notation <math>\{x\}</math> represent <math>\{x[m];\ m\}.</math>

A discrete system transforms an input sequence, <math>\{x\}</math> into an output sequence, <math>\{y\}.</math> In general, every element of the output can depend on every element of the input.  Representing the transformation operator by <math>O</math>, we can write:
<math display="block">y[n] \mathrel{\stackrel{\text{def}}{=}} O_n\{x\}.</math>

Note that unless the transform itself changes with ''n'', the output sequence is just constant, and the system is uninteresting. (Thus the subscript, ''n''.) In a typical system, ''y''[''n''] depends most heavily on the elements of ''x'' whose indices are near ''n''.

For the special case of the [[Kronecker delta function]], <math>x[m] = \delta[m],</math> the output sequence is the '''impulse response''':
<math display="block">h[n] \mathrel{\stackrel{\text{def}}{=}} O_n\{\delta[m];\ m\}.</math>

For a linear system, <math>O</math> must satisfy:
{{NumBlk|:|<math>
  O_n\left\{\sum_{k=-\infty}^{\infty} c_k\cdot x_k[m];\ m\right\} =
  \sum_{k=-\infty}^{\infty} c_k\cdot O_n\{x_k\}.
</math>|{{EquationRef|Eq.4}}}}

And the time-invariance requirement is:
{{NumBlk|:|<math>
\begin{align}
  O_n\{x[m-k];\ m\} &\mathrel{\stackrel{\quad}{=}} y[n-k]\\
                    &\mathrel{\stackrel{\text{def}}{=}} O_{n-k}\{x\}.\,
\end{align}
</math>|{{EquationRef|Eq.5}}}}

In such a system, the impulse response, <math>\{h\}</math>, characterizes the system completely. That is, for any input sequence, the output sequence can be calculated in terms of the input and the impulse response. To see how that is done, consider the identity:
<math display="block">x[m] \equiv \sum_{k=-\infty}^{\infty} x[k] \cdot \delta[m - k],</math>

which expresses <math>\{x\}</math> in terms of a sum of weighted delta functions.

Therefore:
<math display="block">\begin{align}
  y[n] = O_n\{x\}
    &= O_n\left\{\sum_{k=-\infty}^\infty x[k]\cdot \delta[m-k];\ m \right\}\\
    &= \sum_{k=-\infty}^\infty x[k]\cdot O_n\{\delta[m-k];\ m\},\,
\end{align}</math>

where we have invoked {{EquationNote|Eq.4}} for the case <math>c_k = x[k]</math> and <math>x_k[m] = \delta[m-k]</math>.

And because of {{EquationNote|Eq.5}}, we may write:
<math display="block">\begin{align}
  O_n\{\delta[m-k];\ m\} &\mathrel{\stackrel{\quad}{=}} O_{n-k}\{\delta[m];\ m\} \\
                         &\mathrel{\stackrel{\text{def}}{=}} h[n-k].
\end{align}</math>

Therefore:

:{|
| <math>y[n]</math>
| <math>= \sum_{k=-\infty}^{\infty} x[k] \cdot h[n - k]</math>
|-
|
| <math>= \sum_{k=-\infty}^{\infty} x[n-k] \cdot h[k],</math> {{spaces|5}} ([[Convolution#Commutativity|commutativity]])
|}

which is the familiar discrete convolution formula.  The operator <math>O_n</math> can therefore be interpreted as proportional to a weighted average of the function ''x''[''k''].
The weighting function is ''h''[−''k''], simply shifted by amount ''n''. As ''n'' changes, the weighting function emphasizes different parts of the input function.  Equivalently, the system's response to an impulse at ''n''=0 is a "time" reversed copy of the unshifted weighting function.  When ''h''[''k''] is zero for all negative ''k'', the system is said to be [[Causal system|causal]].

=== Exponentials as eigenfunctions ===
An [[eigenfunction]] is a function for which the output of the operator is the same function, scaled by some constant.  In symbols,
<math display="block">\mathcal{H}f = \lambda f ,</math>

where ''f'' is the eigenfunction and <math>\lambda</math> is the [[eigenvalue]], a constant.

The [[exponential function]]s <math>z^n = e^{sT n}</math>, where <math>n \in \mathbb{Z}</math>, are [[eigenfunction]]s of a [[linear]], [[time-invariant]] operator. <math>T \in \mathbb{R}</math> is the sampling interval, and <math>z = e^{sT}, \ z,s \in \mathbb{C}</math>.  A simple proof illustrates this concept.

Suppose the input is <math>x[n] = z^n</math>. The output of the system with impulse response <math>h[n]</math> is then
<math display="block">\sum_{m=-\infty}^{\infty} h[n-m] \, z^m</math>

which is equivalent to the following by the commutative property of [[convolution]]
<math display="block">\sum_{m=-\infty}^{\infty} h[m] \, z^{(n - m)} = z^n \sum_{m=-\infty}^{\infty} h[m] \, z^{-m} = z^n H(z)</math>
where
<math display="block">H(z) \mathrel{\stackrel{\text{def}}{=}} \sum_{m=-\infty}^\infty h[m] z^{-m}</math>
is dependent only on the parameter ''z''.

So <math>z^n</math> is an [[eigenfunction]] of an LTI system because the system response is the same as the input times the constant <math>H(z)</math>.

=== Z and discrete-time Fourier transforms ===
The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems.  The [[Z transform]]
<math display="block">H(z) = \mathcal{Z}\{h[n]\} = \sum_{n=-\infty}^\infty h[n] z^{-n}</math>

is exactly the way to get the eigenvalues from the impulse response.{{clarify|date=September 2020}}  Of particular interest are pure sinusoids; i.e. exponentials of the form <math>e^{j \omega n}</math>, where <math>\omega \in \mathbb{R}</math>.  These can also be written as <math>z^n</math> with <math>z = e^{j \omega}</math>{{clarify|date=September 2020}}. The [[discrete-time Fourier transform]] (DTFT) <math>H(e^{j \omega}) = \mathcal{F}\{h[n]\}</math> gives the eigenvalues of pure sinusoids{{clarify|date=September 2020}}.  Both of <math>H(z)</math> and <math>H(e^{j\omega})</math> are called the ''system function'', ''system response'', or ''transfer function''.

Like the one-sided Laplace transform, the Z transform is usually used in the context of one-sided signals, i.e. signals that are zero for t<0. The discrete-time Fourier transform [[Fourier series]] may be used for analyzing periodic signals.

Due to the convolution property of both of these transforms, the convolution that gives the output of the system can be transformed to a multiplication in the transform domain. That is,
<math display="block">y[n] = (h*x)[n] = \sum_{m=-\infty}^\infty h[n-m] x[m] = \mathcal{Z}^{-1}\{H(z)X(z)\}.</math>

Just as with the Laplace transform transfer function in continuous-time system analysis, the Z transform makes it easier to analyze systems and gain insight into their behavior.

=== Examples ===
{{bulleted list
| A simple example of an LTI operator is the delay operator <math>D\{x[n]\} \mathrel{\stackrel{\text{def}}{=}} x[n-1]</math>.
* <math> D \left( c_1 \cdot x_1[n] + c_2 \cdot x_2[n] \right) = c_1 \cdot x_1[n - 1] + c_2\cdot x_2[n - 1] = c_1\cdot Dx_1[n] + c_2\cdot Dx_2[n]</math> &nbsp; (i.e., it is linear)
* <math> D\{x[n - m]\} = x[n - m - 1] = x[(n - 1) - m] = D\{x\}[n - m]</math> &nbsp; (i.e., it is time invariant)
The Z transform of the delay operator is a simple multiplication by ''z''<sup>−1</sup>. That is,
<math display="block"> \mathcal{Z}\left\{Dx[n]\right\} = z^{-1} X(z). </math>
| Another simple LTI operator is the averaging operator
<math display="block"> \mathcal{A}\left\{x[n]\right\} \mathrel{\stackrel{\text{def}}{=}} \sum_{k=n-a}^{n+a} x[k].</math>

Because of the linearity of sums,
<math display="block">\begin{align}
  \mathcal{A}\left\{c_1 x_1[n] + c_2 x_2[n] \right\}
    &= \sum_{k=n-a}^{n+a} \left( c_1 x_1[k] + c_2 x_2[k] \right)\\
    &= c_1 \sum_{k=n-a}^{n+a} x_1[k] + c_2 \sum_{k=n-a}^{n+a} x_2[k]\\
    &= c_1 \mathcal{A}\left\{x_1[n] \right\} + c_2 \mathcal{A}\left\{x_2[n] \right\},
\end{align}</math>

and so it is linear. Because,
<math display="block">\begin{align}
  \mathcal{A}\left\{x[n-m]\right\}
    &= \sum_{k=n-a}^{n+a} x[k-m]\\
    &= \sum_{k'=(n-m)-a}^{(n-m)+a} x[k']\\
    &= \mathcal{A}\left\{x\right\}[n-m],
\end{align}</math>

it is also time invariant.
}}

=== Important system properties ===
The input-output characteristics of discrete-time LTI system are completely described  by its impulse response <math>h[n]</math>.
Two of the most important properties of a system are causality and stability. Non-causal (in time) systems can be defined and analyzed as above, but cannot be realized in real-time. Unstable systems can also be analyzed and built, but are only useful as part of a larger system whose overall transfer function ''is'' stable.

==== Causality ====
{{Main|Causal system}}
<!--the causal system article needs work-->
A discrete-time LTI system is causal if the current value of the output depends on only the current value and past values of the input.<ref>Phillips 2007, p. 508.</ref>   A necessary and sufficient condition for causality is
<math display="block">h[n] = 0 \ \forall n < 0,</math>
where <math>h[n]</math> is the impulse response.  It is not possible in general to determine causality from the Z transform, because the inverse transform is not unique{{dubious|date=September 2020}}.  When a [[region of convergence]] is specified, then causality can be determined.

==== Stability ====
{{Main|BIBO stability}}
A system is '''bounded input, bounded output stable''' (BIBO stable) if, for every bounded input, the output is finite.  Mathematically, if
<math display="block">\|x[n]\|_{\infty} < \infty</math>

implies that
<math display="block">\|y[n]\|_{\infty} < \infty</math>

(that is, if bounded input implies bounded output, in the sense that the [[Infinity norm|maximum absolute values]] of <math>x[n]</math> and <math>y[n]</math> are finite), then the system is stable.  A necessary and sufficient condition is that <math>h[n]</math>, the impulse response, satisfies
<math display="block">\|h[n]\|_1 \mathrel{\stackrel{\text{def}}{=}} \sum_{n = -\infty}^\infty |h[n]| < \infty.</math>

In the frequency domain, the [[region of convergence]] must contain the [[unit circle]] (i.e., the [[locus (mathematics)|locus]] satisfying <math>|z| = 1</math> for complex ''z'').

==Notes==
{{Reflist}}

== See also ==
* [[Circulant matrix]]
* [[Frequency response]]
* [[Impulse response]]
* [[System analysis]]
* [[Green's function|Green function]]
* [[Signal-flow graph]]

==References==
* {{cite book
  | author=Phillips, C.l., Parr, J.M., & Riskin, E.A
  | title=Signals, systems and Transforms
  | publisher=Prentice Hall
  | year=2007 | isbn=978-0-13-041207-2
  }}
* {{cite book
  | author=Hespanha, J.P.
  | title=Linear System Theory
  | publisher=Princeton university press
  | year=2009| isbn=978-0-691-14021-6
  }}
* {{citation
  | last=Crutchfield | first=Steve
  | url=http://www.jhu.edu/signals/convolve/index.html
  | title=The Joy of Convolution
  | work=Johns Hopkins University
  | date=October 12, 2010
  | access-date=November 21, 2010
  }}
* {{cite journal
  | last1=Vaidyanathan | first1=P. P.
  | last2=Chen | first2=T.
  | title=Role of anticausal inverses in multirate filter banks — Part I: system theoretic fundamentals
  | journal=IEEE Trans. Signal Process.
  | date=May 1995
  | doi=10.1109/78.382395
  | volume=43
  | pages=1090
  | issue=6
  | bibcode = 1995ITSP...43.1090V
  | url=https://authors.library.caltech.edu/6832/1/VAIieeetsp95b.pdf
}}

== Further reading ==
{{Refbegin}}
* {{cite book
| first=Boaz
| last=Porat
| author-link=Boaz Porat
| title=A Course in Digital Signal Processing
| year=1997
| isbn=978-0-471-14961-3
| publisher=John Wiley
| location=New York
}}
* {{cite journal
| last1=Vaidyanathan
| first1=P. P.
| last2=Chen
| first2=T.
| title=Role of anticausal inverses in multirate filter banks — Part I: system theoretic fundamentals
| journal=IEEE Trans. Signal Process.
| date=May 1995
| doi=10.1109/78.382395
| volume=43
| pages=1090
| issue=5
| bibcode = 1995ITSP...43.1090V
| url=https://authors.library.caltech.edu/6832/1/VAIieeetsp95b.pdf
}}
{{Refend}}

== External links ==
* [http://www.tedpavlic.com/teaching/osu/ece209/support/circuits_sys_review.pdf ECE 209: Review of Circuits as LTI Systems]&nbsp;– Short primer on the mathematical analysis of (electrical) LTI systems.
* [http://www.tedpavlic.com/teaching/osu/ece209/lab3_opamp_FO/lab3_opamp_FO_phase_shift.pdf ECE 209: Sources of Phase Shift]&nbsp;– Gives an intuitive explanation of the source of phase shift in two common electrical LTI systems.
* [http://www.ece.jhu.edu/~cooper/courses/214/signalsandsystemsnotes.pdf JHU 520.214 Signals and Systems course notes]. An encapsulated course on LTI system theory. Adequate for self teaching.
* [http://www.etti.unibw.de/labalive/tutorial/lti/ LTI system example: RC low-pass filter]. Amplitude and phase response.

{{Authority control}}

{{DEFAULTSORT:Lti System Theory}}
[[Category:Digital signal processing]]
[[Category:Electrical engineering]]
[[Category:Classical control theory]]
[[Category:Signal processing]]
[[Category:Frequency-domain analysis]]
[[Category:Time domain analysis]]