Short-Wave Solar Radiation in the Earth’s Atmosphere Part 6 pdf

Accounting for Measurement Uncertainties and Regularization of the Solution 149

Observational data Y contain the random errors characterized with the SD

of components yi, i = 1, . . ., N. In general, the errors could correlate, i. e. they

are interconnected (although everybody aims to avoid this correlation with all

possible means in practice). Thus, the observational errors are described with

symmetric covariance matrix SY of dimension N × N, which can be obtained

conveniently by writing schematically according to Anderson (1971) as:

SY =

(Y − Y¯ )(Y − Y¯ )

+ , (4.37)

where Y¯ is the exact (unknown) value of the measured vector, Y is the observed

value of the vector (distinguishing from the exact value owing to the observa￾tional errors), the summation is understood as an averaging over all statistical

realizations of the observations of the random vector (over the general set).

The relation for covariance matrix of the errors SX of parameters X, of

dimension K ×K written in the same way as (4.37). Then, substituting relation

(4.36) to it, the following is obtained:

SX =

(AY − AY¯ )(AY − AY¯ )

+ = A

(Y − Y¯ )(Y − Y¯ )

+

A+ ,

SX = ASYA+ .

(4.38)

A set of important consequences directly follows from (4.38)

Consequence 1. Equation (4.38) expresses the relationship between the co￾variance matrices of observational errors Y and parameters X linearly linked

with them through (4.36), i. e. allows the finding of errors of the calculated

parameters from the known observational errors. Namely, values √(SX)kk are

the SD of parameters xk, values (SX)kj|

(SX)kk(SX)jj are the coefficients of the

correlation between the uncertainties of parameters xk and xj. In the particular

case of non-correlated observational errors that is often met in practice, (4.38)

converts to the explicit formula convenient for calculations:

(SX)kj =



N

i=1

akiajis

2

i , k = 1, . . ., K , j = 1, . . ., K , (4.39)

where aki are the elements of matrix A, si is the SD of parameter yi. In the

case of the equally accurate measurements, i. e. s = s1 = ... = sN, the direct

proportionality of the SD of the observations and parameters follows from

(4.39):

(SX)kj = s

2

N

i=1

akiaji .

Consequence 2. From the derivation of (4.38) the general set could be evi￾dently replaced with a finite sample from M measurements Y(m)

, m = 1, . . ., M,

150 The Problem of Retrieving Atmospheric Parameters from Radiative Observations

i. e. SY in (4.37) is obtained as an estimation of the covariance matrix using the

known formulas:

(SY )ij = 1

M − 1



M

m=1

(y(m)

i − y¯i)(y

(m)

j − y¯j) , y¯i = 1

M



M

m=1

y(m)

i ,

i = 1, . . ., N , j = 1, . . ., N .

Then the analogous estimations are inferred for matrix SX with (4.38). On

the one hand, if just random observational errors are implied, then all M

measurements will relate to one real magnitude of the measured value. But

on the other hand the elements of matrix SY could be treated more widely,

as characteristics of variations of the vector Y components caused not by

the random errors only but by any changes of the measured value. In this

case, (4.38) is the estimation of the variations of parameters X by the known

variations of values Y

Consequence 3. Consider the simplest case of the relations similar to (4.36)

– the calculation of the mean value over all components of vector Y i. e. x = 1

N

N

i=1 yi (here K = 1, so value X is specified as a scalar). Then aki = 1|N for

all numbers i and the following is derived from (4.38) for the SD of value x:

s(x) = 1

N

 

N

i=1



N

j=1

(SY )ij . (4.40)

For the non-correlated observational errors in sum (4.40) only the diagonal

terms of the matrix remain and it transforms to the well-known errors sum￾mation rule:

s(x) = 1

N

 

N

i=1

(SY )ii . (4.41)

SD of the mean value decreases with the increasing of the quantity of the av￾eraged values as √N (for the equally accurate measurements s(x) = s(y)|

√N),

as per (4.41). As not only the uncertainties of the direct measurements could

be implied under SY , the properties of (4.40) and (4.41) are often used dur￾ing the interpretation of inverse problem solutions of atmospheric optics. For

example, after solving the inverse problem the passage from the optical char￾acteristics of thin layers to the optical characteristics of rather thick layers or

of the whole atmospheric column essentially diminishes the uncertainty of

the obtained results (Romanov et al. 1989). Note also that we have used the

relations similar to (4.41) in Sect. 2.1 while deriving the expressions for the

irradiances dispersion (2.17) in the Monte-Carlo method.

Consequence 4. Analyzing (4.41) it is necessary to mention one other obsta￾cle. It is written for the real numbers, but any presentation of the observational

Accounting for Measurement Uncertainties and Regularization of the Solution 151

results has a discrete character in reality, i. e. it corresponds finally to inte￾gers. The discreteness becomes apparent in an uncertainty of the process of

the instrument reading. Hence, real dispersion s(x) could not be diminished

infinitely, even if N → ∞ [indeed the length value measured by the ruler

with the millimeter scale evidently can’t be obtained with the accuracy 1 µm

even after a million measurements, although it does follow from (4.41)]. Re￾gretfully, not enough attention is granted to the question of influence of the

measurement discreteness on the result processing in the literature. The book

by Otnes and Enochson (1978) could be mentioned as an exception. However,

this phenomenon is well known in practice of computer calculations where the

word length is finite too. It leads to an accumulation of computer uncertain￾ties of calculations, and special algorithms are to be used for diminishing this

influence even during the simplest calculation of the arithmetic mean value (!)

(Otnes and Enochson 1978). As per this brief analysis, the discreteness causes

the underestimation of the real uncertainties of the averaged values.

Consequence 5. In addition to the considered averaging, the interpolation,

numerical differentiation, and integration are the often-met operations similar

to (4.36). Actually, they are all reduced to certain linear transformations of

value yi and could be easily written in the matrix form (4.36). Thus, (4.38)

is a solution of the problem of uncertainty finding during the operations of

interpolation, numerical differentiation, and integration of the results. Note

that in the general case the mentioned uncertainties will correlate even if the

initial observational uncertainties are independent.

Consequence 6. Matrix SX does not depend on vectorA0 in (4.36). Assuming

A0 = AY0, where Y0 is the certain vector consisting of the constants, (4.38)

turns out valid not for the initial vector only but for any Y + Y0 vector, i. e.

the covariance error matrix of parameters vector X does not depend on the

addition of any constant to observation vector Y.

Consequence 7. Consider nonlinear dependence X = A(Y). It could be re￾duced to the above-described linear relationship (4.36) using linearization, i. e.

expanding A(Y) into Taylor series around a concrete value of Y and accounting

only for the linear terms as shown in the previous section. Then the elements

of matrix A will be partial derivatives aki = ∂(A(Y))k|∂yi, all constant terms

as per consequence 6 will not influence the uncertainty estimations and the

same formula as (4.38) will be obtained. For example, the uncertainties of the

surface albedo have been calculated in this way with the covariance matrix of

the irradiance uncertainties obtained at the second stage of the processing of

the sounding results in Sect. 3.3. The uncertainties of the retrieved parameters,

while solving the inverse problem in the case of the overcast sky have been

calculated in this way, as will be considered in Chap. 6. Note, that relation (4.38)

is an approximate estimation of the parameters of uncertainty in the nonlinear

case because for exact estimation all terms of Taylor series are to be accounted.

The accuracy of this estimation is higher if the observational uncertainties (i. e.

the matrix SX elements are less).

Return to the inverse problem solution and to begin with again consider the

case of the linear relationship of observational results Y and desired parame￾ters X (4.9): Y˜ = G0 + GX. Let the observational errors obey the law of normal