首页资源分类电子电路数字电路 > 随机信号分析

随机信号分析

已有 451217个资源

下载专区

上传者其他资源

文档信息举报收藏

标    签:随机信号分析

分    享:

文档简介

随机信号分析英文版教材

文档预览

Introduction to Random Signals and Noise Wim C. van Etten University of Twente, The Netherlands Introduction to Random Signals and Noise Introduction to Random Signals and Noise Wim C. van Etten University of Twente, The Netherlands Copyright # 2005 John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England Telephone (+44) 1243 779777 Email (for orders and customer service enquiries): cs-books@wiley.co.uk Visit our Home Page on www.wiley.com All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher. Requests to the Publisher should be addressed to the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or emailed to permreq@wiley.co.uk, or faxed to (+44) 1243 770620. Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The Publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the Publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. Other Wiley Editorial Offices John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA Wiley-VCH Verlag GmbH, Boschstr. 12, D-69469 Weinheim, Germany John Wiley & Sons Australia Ltd, 42 McDougall Street, Milton, Queensland 4064, Australia John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop # 02-01, Jin Xing Distripark, Singapore 129809 John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1 Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library ISBN-13 978-0-470-02411-9 (HB) ISBN-10 0-470-02411-9 (HB) Typeset in 10/12pt Times by Thomson Press (India) Limited, New Delhi Printed and bound in Great Britain by Antony Rowe Ltd, Chippenham, Wiltshire This book is printed on acid-free paper responsibly manufactured from sustainable forestry in which at least two trees are planted for each one used for paper production. To Kitty, to Sascha, Anne and Emmy, to Bjo¨rn and Esther Contents Preface xi 1 Introduction 1 1.1 Random Signals and Noise 1 1.2 Modelling 1 1.3 The Concept of a Stochastic Process 2 1.3.1 Continuous Stochastic Processes 4 1.3.2 Discrete-Time Processes (Continuous Random Sequences) 5 1.3.3 Discrete Stochastic Processes 6 1.3.4 Discrete Random Sequences 7 1.3.5 Deterministic Function versus Stochastic Process 8 1.4 Summary 8 2 Stochastic Processes 9 2.1 Stationary Processes 9 2.1.1 Cumulative Distribution Function and Probability Density Function 9 2.1.2 First-Order Stationary Processes 10 2.1.3 Second-Order Stationary Processes 11 2.1.4 Nth-Order Stationary Processes 11 2.2 Correlation Functions 11 2.2.1 The Autocorrelation Function, Wide-Sense Stationary Processes and Ergodic Processes 11 2.2.2 Cyclo-Stationary Processes 16 2.2.3 The Cross-Correlation Function 19 2.2.4 Measuring Correlation Functions 24 2.2.5 Covariance Functions 26 2.2.6 Physical Interpretation of Process Parameters 27 2.3 Gaussian Processes 27 2.4 Complex Processes 30 2.5 Discrete-Time Processes 31 2.5.1 Mean, Correlation Functions and Covariance Functions 31 2.6 Summary 33 2.7 Problems 34 3 Spectra of Stochastic Processes 39 3.1 The Power Spectrum 39 viii CONTENTS 3.2 The Bandwidth of a Stochastic Process 43 3.3 The Cross-Power Spectrum 45 3.4 Modulation of Stochastic Processes 47 3.4.1 Modulation by a Random Carrier 49 3.5 Sampling and Analogue-To-Digital Conversion 50 3.5.1 Sampling Theorems 51 3.5.2 A/D Conversion 54 3.6 Spectrum of Discrete-Time Processes 57 3.7 Summary 58 3.8 Problems 59 4. Linear Filtering of Stochastic Processes 65 4.1 Basics of Linear Time-Invariant Filtering 65 4.2 Time Domain Description of Filtering of Stochastic Processes 68 4.2.1 The Mean Value of the Filter Output 68 4.2.2 The Autocorrelations Function of the Output 69 4.2.3 Cross-Correlation of the Input and Output 70 4.3 Spectra of the Filter Output 71 4.4 Noise Bandwidth 74 4.4.1 Band-Limited Processes and Systems 74 4.4.2 Equivalent Noise Bandwidth 75 4.5 Spectrum of a Random Data Signal 77 4.6 Principles of Discrete-Time Signals and Systems 82 4.6.1 The Discrete Fourier Transform 82 4.6.2 The z-Transform 86 4.7 Discrete-Time Filtering of Random Sequences 90 4.7.1 Time Domain Description of the Filtering 90 4.7.2 Frequency Domain Description of the Filtering 91 4.8 Summary 93 4.9 Problems 94 5 Bandpass Processes 101 5.1 Description of Deterministic Bandpass Signals 101 5.2 Quadrature Components of Bandpass Processes 106 5.3 Probability Density Functions of the Envelope and Phase of Bandpass Noise 111 5.4 Measurement of Spectra 115 5.4.1 The Spectrum Analyser 115 5.4.2 Measurement of the Quadrature Components 118 5.5 Sampling of Bandpass Processes 119 5.5.1 Conversion to Baseband 119 5.5.2 Direct Sampling 119 5.6 Summary 121 5.7 Problems 121 6 Noise in Networks and Systems 129 6.1 White and Coloured Noise 129 6.2 Thermal Noise in Resistors 130 6.3 Thermal Noise in Passive Networks 131 CONTENTS ix 6.4 System Noise 137 6.4.1 Noise in Amplifiers 138 6.4.2 The Noise Figure 140 6.4.3 Noise in Cascaded systems 142 6.5 Summary 146 6.6 Problems 146 7 Detection and Optimal Filtering 153 7.1 Signal Detection 154 7.1.1 Binary Signals in Noise 154 7.1.2 Detection of Binary Signals in White Gaussian Noise 158 7.1.3 Detection of M-ary Signals in White Gaussian Noise 161 7.1.4 Decision Rules 165 7.2 Filters that Maximize the Signal-to-Noise Ratio 165 7.3 The Correlation Receiver 171 7.4 Filters that Minimize the Mean-Squared Error 175 7.4.1 The Wiener Filter Problem 175 7.4.2 Smoothing 176 7.4.3 Prediction 179 7.4.4 Discrete-Time Wiener Filtering 183 7.5 Summary 185 7.6 Problems 185 8 Poisson Processes and Shot Noise 193 8.1 Introduction 193 8.2 The Poisson Distribution 194 8.2.1 The Characteristic Function 194 8.2.2 Cumulants 196 8.2.3 Interarrival Time and Waiting Time 197 8.3 The Homogeneous Poisson Process 198 8.3.1 Filtering of Homogeneous Poisson Processes and Shot Noise 199 8.4 Inhomogeneous Poisson Processes 204 8.5 The Random-Pulse Process 205 8.6 Summary 207 8.7 Problems 208 References 211 Further Reading 213 Appendices 215 A. Representation of Signals in a Signal Space 215 A.1 Linear Vector Spaces 215 A.2 The Signal Space Concept 216 A.3 Gram–Schmidt Orthogonalization 218 A.4 The Representation of Noise in Signal Space 219 A.4.1 Relevant and Irrelevant Noise 221 A.5 Signal Constellations 222 A.5.1 Binary Antipodal Signals 222 x CONTENTS A.5.2 Binary Orthogonal Signals 223 A.5.3 Multiphase Signals 224 A.5.4 Multiamplitude Signals 224 A.5.5 QAM Signals 225 A.5.6 M-ary Orthogonal Signals 225 A.5.7 Biorthogronal Signals 225 A.5.8 Simplex Signals 226 A.6 Problems 227 B. Attenuation, Phase Shift and Decibels 229 C. Mathematical Relations 231 C.1 Trigonometric Relations 231 C.2 Derivatives 232 C.2.1 Rules fn Differentiation 232 C.2.1 Chain Rule 232 C.2.3 Stationary Points 233 C.3 Indefinite Integrals 233 C.3.1 Basic Integrals 233 C.3.2 Integration by Parts 234 C.3.3 Rational Algebraic Functions 234 C.3.4 Trigonometric Functions 235 C.3.5 Exponential Functions 236 C.4 Definite Integrals 236 C.5 Series 237 C.6 Logarithms 238 D. Summary of Probability Theory 239 E. Definition of a Few Special Functions 241 F. The Q(.) and erfc Function 243 G. Fourier Transforms 245 H. Mathematical and Physical Constants 247 Index 249 Preface Random signals and noise are present in several engineering systems. Practical signals seldom lend themselves to a nice mathematical deterministic description. It is partly a consequence of the chaos that is produced by nature. However, chaos can also be man-made, and one can even state that chaos is a conditio sine qua non to be able to transfer information. Signals that are not random in time but predictable contain no information, as was concluded by Shannon in his famous communication theory. To deal with this randomness we have to nevertheless use a characterization in deterministic terms; i.e. we employ probability theory to determine characteristic descriptions such as mean, variance, correlation, etc. Whenever chaotic behaviour is timedependent, as is often the case for random signals, the time parameter comes into the picture. This calls for an extension of probability theory, which is the theory of stochastic processes and random signals. With the involvement of time, the phenomenon of frequency also enters the picture. Consequently, random signal theory leans heavily on both probability and Fourier theories. Combining these subjects leads to a powerful tool for dealing with random signals and noise. In practice, random signals may be encountered as a desired signal such as video or audio, or it may be an unwanted signal that is unintentionally added to a desired (information bearing) signal thereby disturbing the latter. One often calls this unwanted signal noise. Sometimes the undesired signal carries unwanted information and does not behave like noise in the classical sense. In such cases it is termed as interference. While it is usually difficult to distinguish (at least visually) between the desired signal and noise (or interference), by means of appropriate signal processing such a distinction can be made. For example, optimum receivers are able to enhance desired signals while suppressing noise and interference at the same time. In all cases a description of the signals is required in order to be able to analyse their impact on the performance of the system under consideration. In communication theory this situation often occurs. The random time-varying character of signals is usually difficult to describe, and this is also true for associated signal processing activities such as filtering. Nevertheless, there is a need to characterize these signals using a few deterministic parameters that allow a system user to assess system performance. This book deals with stochastic processes and noise at an introductory level. Probability theory is assumed to be known. The same holds for mathematical background in differential and integral calculus, Fourier analysis and some basic knowledge of network and linear system theory. It introduces the subject in the form of theorems, properties and examples. Theorems and important properties are placed in frames, so that the student can easily xii PREFACE summarize them. Examples are mostly taken from practical applications. Each chapter concludes with a summary and a set of problems that serves as practice material. The book is well suited for dealing with the subject at undergraduate level. A few subjects can be skipped if they do not fit into a certain curriculum. Besides, the book can also serve as a reference for the experienced engineer in his daily work. In Chapter 1 the subject is introduced and the concept of a stochastic process is presented. Different types of processes are defined and elucidated by means of simple examples. Chapter 2 gives the basic definitions of probability density functions and includes the time dependence of these functions. The approach is based on the ‘ensemble’ concept. Concepts such as stationarity, ergodicity, correlation functions and covariance functions are introduced. It is indicated how correlation functions can be measured. Physical interpretation of several stochastic concepts are discussed. Cyclo-stationary and Gaussian processes receive extra attention, as they are of practical importance and possess some interesting and convenient properties. Complex processes are defined analogously to complex variables. Finally, the different concepts are reconsidered for discrete-time processes. In Chapter 3 a description of stochastic processes in the frequency domain is given. This results in the concept of power spectral density. The bandwidth of a stochastic process is defined. Such an important subject as modulation of stochastic processes is presented, as well as the synchronous demodulation. In order to be able to define and describe the spectrum of discrete-time processes, a sampling theorem for these processes is derived. After the basic concepts and definitions treated in the first three chapters, Chapter 4 starts with applications. Filtering of stochastic processes is the main subject of this chapter. We confine ourselves to linear, time-invariant filtering and derive both the correlation functions and spectra of a two-port system. The concept of equivalent noise bandwidth has been defined in order to arrive at an even more simple description of noise filtering in the frequency domain. Next, the calculation of the spectrum of random data signals is presented. A brief resume´ of the principles of discrete-time signals and systems is dealt with using the z-transform and discrete Fourier transform, based on which the filtering of discrete-time processes is described both in time and frequency domains. Chapter 5 is devoted to bandpass processes. The description of bandpass signals and systems in terms of quadrature components is introduced. The probability density functions of envelope and phase are derived. The measurement of spectra and operation of the spectrum analyser is discussed. Finally, sampling and conversion to baseband of bandpass processes is discussed. Thermal noise and its impact on systems is the subject of Chapter 6. After presenting the spectral densities we consider the role of thermal noise in passive networks. System noise is considered based on the thermal noise contribution of amplifiers, the noise figure and the influence of cascading of systems on noise performance. Chapter 7 is devoted to detection and optimal filtering. The chapter starts by considering hypothesis testing, which is applied to the detection of a binary signal disturbed by white Gaussian noise. The matched filter emerges as the optimum filter for optimum detection performance. Finally, filters that minimize the mean squared error (Wiener filters) are derived. They can be used for smoothing stored data or portions of a random signal that arrived in the past. Filters that produce an optimal prediction of future signal values can also be designed. Finally, Chapter 8 is of a more advanced nature. It presents the basics of random point processes, of which the Poisson process is the most well known. The characteristic function PREFACE xiii plays a crucial role in analysing these processes. Starting from that process several shot noise processes are introduced: the homogeneous Poisson process, the inhomogeneous Poisson process, the Poisson impulse process and the random-pulse process. Campbell’s theorem is derived. A few application areas of random point processes are indicated. The appendices contain a few subjects that are necessary for the main material. They are: signal space representation and definitions of attenuation, phase shift and decibels. The rest of the appendices comprises basic mathematical relations, a summary of probability theory, definitions of special functions, a list and properties of Fourier transform pairs, and a few mathematical and physical constants. Finally, I would like to thank those people who contributed in one way or another to this text. My friend Rajan Srinivasan provided me with several suggestions to improve the content. Also, Arjan Meijerink carefully read the draft and made suggestions for improvement. Last but certainly not least, I thank my wife Kitty, who allowed me to spend so many hours of our free time to write this text. Wim van Etten Enschede, The Netherlands 1 Introduction 1.1 RANDOM SIGNALS AND NOISE In (electrical) engineering one often encounters signals that do not have a precise mathematical description, since they develop as random functions of time. Sometimes this random development is caused by a single random variable, but often it is a consequence of many random variables. In other cases the causes of randomness are not clear and a description is not possible, but the signal is characterized by means of measurements only. A random time function may be a desired signal, such as an audio or video signal, or it may be an unwanted signal that is unintentionally added to a desired (information) signal and disturbs the desired signal. We call the desired signal a random signal and the unwanted signal noise. However, the latter often does not behave like noise in the classical sense, but it is more like interference. Then it is an information bearing signal as well, but undesired. A desired signal and noise (or interference) can, in general, not be distinguished completely; by means of well-defined signal processing in a receiver, the desired signal may be favoured in a maximal way whereas the disturbance is suppressed as much as possible. In all cases a description of the signals is required in order to be able to analyse its impact on the performance of the system under consideration. Especially in communication theory this situation often occurs. The random character as a function of time makes the signals difficult to describe and the same holds for signal processing or filtering. Nevertheless, there is a need to characterize these signals by a few deterministic parameters that enable the system user to assess the performance of the system. The tool to deal with both random signals and noise is the concept of the stochastic process, which is introduced in Section 1.3. This book gives an elementary introduction to the methods used to describe random signals and noise. For that purpose use is made of the laws of probability, which are extensively described in textbooks [1–5]. 1.2 MODELLING When studying and analysing random signals one is mainly committed to theory, which however, can be of good predictive value. Actually, the main activity in the field of random signals is modelling of processes and systems. Many scientists and engineers have Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 2 INTRODUCTION PHYSICAL SYSTEM PHYSICAL MODEL MATH. MODEL MATH. CALCUL. MEASUREMENT COMPARISON INTERPRETATION RESULT Figure 1.1 The process of modelling contributed to that activity in the past and their results have been checked in practice. When a certain result agrees (at least to a larger extent) with practical measurements, then there is confidence in and acceptance of the result for practical application. This process of modelling has schematically been depicted in Figure 1.1. In the upper left box of this scheme there is the important physical process. Based on our knowledge of the physics of this process we make a physical model of it. This physical model is converted into a mathematical model. Both modelling activities are typical engineer tasks. In this mathematical model the physics is no longer formally recognized, but the laws of physics will be included with their mathematical description. Once the mathematical model has been completed and the questions are clear we can forget about the physics for the time being and concentrate on doing the mathematical calculations, which may help us to find the answers to our questions. In this phase the mathematicians can help the engineer a lot. Let us suppose that the mathematical calculations give a certain outcome, or maybe several outcomes. These outcomes would then need to be interpreted in order to discover what they mean from a physical point of view. This ends the role of the mathematician, since this phase is maybe the most difficult engineering part of the process. It may happen that certain mathematical solutions have to be discarded since they contradict physical laws. Once the interpretation has been completed there is a return to the physical process, as the practical applicability of the results needs to be checked. In order to check these the quantities or functions that have been calculated are measured. The measurement is compared to the calculated result and in this way the physical model is validated. This validation may result in an adjustment of the physical model and another cycle in the loop is made. In this way the model is refined iteratively until we are satisfied about the validation. If there is a shortage of insight into the physical system, so that the physical model is not quite clear, measurements of the physical system may improve the physical model. In the courses that are taught to students, models that have mainly been validated in this way are presented. However, it is important that students are aware of this process and the fact that the models that are presented may be a result of a difficult struggle for many years by several physicists, engineers and mathematicians. Sometimes students are given the opportunity to be involved in this process during research assignments. 1.3 THE CONCEPT OF A STOCHASTIC PROCESS In probability theory a random variable is a rule that assigns a number to every outcome of an experiment, such as, for example, rolling a die. This random variable X is associated with a sample space S, such that according to a well-defined procedure to each event s in the THE CONCEPT OF A STOCHASTIC PROCESS 3 ... xn +2(t ) xn +1(t ) xn (t ) xn –1(t ) 0 ... t X(t 1) Figure 1.2 A few sample functions of a stochastic process sample space a number is assigned to X and is denoted by XðsÞ. For stochastic processes, on the other hand, a time function xðt; sÞ is assigned to every outcome in the sample space. Within the framework of the experiment the family (or ensemble) of all possible functions that can be realized is called the stochastic process and is denoted by Xðt; sÞ. A specific waveform out of this family is denoted by xnðtÞ and is called a sample function or a realization of the stochastic process. When a realization in general is indicated the subscript n is omitted. Figure 1.2 shows a few sample functions that are supposed to constitute an ensemble. The figure gives an example of a finite number of possible realizations, but the ensemble may consist of an infinite number of realizations. The realizations may even be uncountable. A realization itself is sometimes called a stochastic process as well. Moreover, a stochastic process produces a random variable that arises from giving t a fixed value with s being variable. In this sense the random variable Xðt1; sÞ ¼ Xðt1Þ is found by considering the family of realizations at the fixed point in time t1 (see Figure 1.2). Instead of Xðt1Þ we will also use the notation X1. The random variable X1 describes the statistical properties of the process at the instant of time t1. The expectation of X1 is called the ensemble mean or the expected value or the mean of the stochastic process (at the instant of time t1). Since t1 may be arbitrarily chosen, the mean of the process will in general not be constant, i.e. it may have different values for different values of t. Finally, a stochastic process may represent a single number by giving both t and s fixed values. The phrase ‘stochastic process’ may therefore have four different interpretations. They are: 1. A family (or ensemble) of time functions. Both t and s are variables. 2. A single time function called a sample function or a realization of the stochastic process. Then t is a variable and s is fixed. 3. A random variable; t is fixed and s is variable. 4. A single number; both t and s are fixed. 4 INTRODUCTION Which of these four interpretations holds in a specific case should follow from the context. Different classes of stochastic processes may be distinguished. They are classified on the basis of the characteristics of the realization values of the process x and the time parameter t. Both can be either continuous or discrete, in any combination. Based on this we have the following classes:  Both the values of XðtÞ and the time parameter t are continuous. Such a process is called a continuous stochastic process.  The values of XðtÞ are continuous, whereas time t is discrete. These processes are called discrete-time processes or continuous random sequences. In the remainder of the book we will use the term discrete-time process.  If the values of XðtÞ are discrete but the time axis is continuous, we call the process a discrete stochastic process.  Finally, if both the process values and the time scale are discrete, we say that the process is a discrete random sequence. In Table 1.1 an overview of the different classes of processes is presented. In order to get some feeling for stochastic processes we will consider a few examples. XðtÞ Continuous Discrete Table 1.1 Summary of names of different processes Continuous Time Discrete Continuous stochastic process Discrete stochastic process Discrete-time process Discrete random sequence 1.3.1 Continuous Stochastic Processes For this class of processes it is assumed that in principle the following holds: À1 < xðtÞ < 1 and À1 < t < 1 ð1:1Þ An example of this class was already given by Figure 1.2. This could be an ensemble of realizations of a thermal noise process as is, for instance, produced by a resistor, the characteristics of which are to be dealt with in Chapter 6. The underlying experiment is selecting a specific resistor from a collection of, let us say, 100  resistors. The voltage across every selected resistor corresponds to one of the realizations in the figure. Another example is given below. Example 1.1: The process we consider now is described by the equation XðtÞ ¼ cosð!0t À ÂÞ ð1:2Þ THE CONCEPT OF A STOCHASTIC PROCESS 5 ...... ...... t Figure 1.3 Ensemble of sample functions of the stochastic process cosð!0t À ÂÞ, with  uniformly distributed on the interval ð0; 2Š with !0 a constant and  a random variable with a uniform probability density function on the interval ð0; 2Š. In this example the set of realizations is in fact uncountable, as  assumes continuous values. The ensemble of sample functions is depicted in Figure 1.3. Thus each sample function consists of a cosine function with unity amplitude, but the phase of each sample function differs randomly from others. For each sample function a drawing is taken from the uniform phase distribution. We can imagine this process as follows. Consider a production process of crystal oscillators, all producing the same amplitude unity and the same radial frequency !0. When all those oscillators are switched on, their phases will be mutually independent. The family of all measured output waveforms can be considered as the ensemble that has been presented in Figure 1.3. This process will get further attention in different chapters that follow. & 1.3.2 Discrete-Time Processes (Continuous Random Sequences) The description of this class of processes becomes more and more important due to the increasing use of modern digital signal processors which offer flexibility and increasing speed and computing power. As an example of a discrete-time process we can imagine sampling the process that was given in Figure 1.2. Let us suppose that to this process ideal sampling is applied at equidistant points in time with sampling period Ts; with ideal sampling we mean the sampling method where the values at Ts are replaced by delta functions of amplitude XðnTsÞ [6]. However, to indicate that it is now a discrete-time process we denote it by X½nŠ, where n is an integer running in principle from À1 to þ1. We know from the sampling theorem (see Section 3.5.1 or, for instance, references [1] and [7]) that the original signal can perfectly be recovered from its samples, provided that the signals are band-limited. The process that is produced in this way is given in Figure 1.4, where the sample values are presented by means of the length of the arrows. ... 6 INTRODUCTION Ts xn +2[n] xn +1[n] xn [n] ... xn –1[n] 0 n Figure 1.4 Example of a discrete-time stochastic process Another important example of the discrete-time process is the so-called Poisson process, where there are no equidistant samples in time but the process produces ‘samples’ at random points in time. This process is an adequate model for shot noise and it is dealt with in Chapter 8. 1.3.3 Discrete Stochastic Processes In this case the time is continuous and the values discrete. We present two examples of this class. The second one, the random data signal, is of great practical importance and we will consider it in further detail in Chapter 4. Example 1.2: This example is a very simple one. The ensemble of realizations consists of a set of constant time functions. According to the outcome of an experiment one of these constants may be chosen. This experiment can be, for example, the rolling of a die. In that case the number of realizations can be six ðn ¼ 6Þ, equal to the usual number of faces of a die. Each of the outcomes s 2 f1; 2; 3; 4; 5; 6g has a one-to-one correspondence to one of these numbered constant functions of time. The ensemble is depicted in Figure 1.5. & Example 1.3: Another important stochastic process is the random data signal. It is a signal that is produced by many data sources and is described by X XðtÞ ¼ Anpðt À nT À ÂÞ ð1:3Þ n THE CONCEPT OF A STOCHASTIC PROCESS 7 xn (t ) . . . . x 3(t ) x 2(t ) 0 t x 1(t ) Figure 1.5 Ensemble of sample functions of the stochastic process constituted by a number of constant time functions where fAng are the data bits that are randomly chosen from the set An 2 fþ1; À1g. The rectangular pulse pðtÞ of width T serves as the carrier of the information. Now  is supposed to be uniformly distributed on the bit interval ð0; TŠ, so that all data sources of the family have the same bit period, but these periods are not synchronized. The ensemble is given in Figure 1.6. & 1.3.4 Discrete Random Sequences The discrete random sequence can be imagined to result from sampling a discrete stochastic process. Figure 1.7 shows the result of sampling the random data signal from Example 1.3. We will base the further development of the concept, description and properties of stochastic processes on the continuous stochastic process. Then we will show how these are extended to discrete-time processes. The two other classes do not get special attention, but T ..... t P Figure 1.6 Ensemble of sample functions of the stochastic process n Anpðt À nT À ÂÞ, with  uniformly distributed on the interval ð0; TŠ ... ... 8 INTRODUCTION xn +2[n] xn +1[n] xn [n] xn –1[n] xn –2[n] n Figure 1.7 Example of a discrete random sequence are considered as special cases of the former ones by limiting the realization values x to a discrete set. 1.3.5 Deterministic Function versus Stochastic Process The concept of the stochastic process does not conflict with the theory of deterministic functions. It should be recognized that a deterministic function can be considered as nothing else but a special case of a stochastic process. This is elucidated by considering Example 1.1. If the random variable  is given the probability density function fÂðÞ ¼ ðÞ, then the stochastic process reduces to the function cosð!0tÞ. The given probability density function is actually a discrete one with a single outcome. In fact, the ensemble of the process reduces in this case to a family comprising merely one member. This is a general rule; when the probability density function of the stochastic process that is governed by a single random variable consists of a single delta function, then a deterministic function results. This way of generalization avoids the often confusing discussion on the difference between a deterministic function on the one hand and a stochastic process on the other hand. In view of the consideration presented here they can actually be considered as members of the same class, namely the class of stochastic processes. 1.4 SUMMARY Definitions of random signals and noise have been given. A random signal is, as a rule, an information carrying wanted signal that behaves randomly. Noise also behaves randomly but is unwanted and disturbs the signal. A common tool to describe both is the concept of a stochastic process. This concept has been explained and different classes of stochastic processes have been identified. They are distinguished by the behaviour of the time parameter and the values of the process. Both can either be continuous or discrete. 2 Stochastic Processes In this chapter some basic concepts known from probability theory will be extended to include the time parameter. It is the time parameter that makes the difference between a random variable and a stochastic process. The basic concepts are: probability density function and correlation. The time dependence of the signals asks for a few new concepts, such as the correlation function, stationarity and ergodicity. 2.1 STATIONARY PROCESSES As has been indicated in the introduction chapter we can fix the time parameter of a stochastic process. In this way we have a random variable, which can be characterized by means of a few deterministic numbers such as the mean, variance, etc. These quantities are defined using the probability density function. When fixing two time parameters we can consider two random variables simultaneously. Here also we can define joint random variables and, related to that, characterize quantities using the joint probability density function. In this way we can proceed, in general, to the case of N variables that are described by an N-dimensional joint probability density function, with N an arbitrary number. Roughly speaking we can say that a stochastic process is stationary if its statistical properties do not depend on the time parameter. This rough definition will be elaborated in more detail in the rest of this chapter. There are several types of stationarity and for the main types we will present exact definitions in the sequel. 2.1.1 Cumulative Distribution Function and Probability Density Function In order to be able to define stationarity, the probability distribution and density functions as they are applied to the stochastic process XðtÞ have to be defined. For a fixed value of time parameter t1 the cumulative probability distribution function or, for short, distribution function is defined by FXðx1; t1Þ ¼4 PfXðt1Þ x1g ð2:1Þ Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 10 STOCHASTIC PROCESSES From this notation it follows that FX may be a function of the value of t1 that has been chosen. For two random variables X1 ¼ Xðt1Þ and X2 ¼ Xðt2Þ we introduce the two-dimensional extension of Equation (2.1): FXðx1; x2; t1; t2Þ ¼4 PfXðt1Þ x1; Xðt2Þ x2g ð2:2Þ the second-order, joint probability distribution function. In an analogous way we denote the Nth-order, joint probability distribution function FXðx1; . . . ; xN ; t1; . . . ; tNÞ ¼4 PfXðt1Þ x1; . . . ; XðtN Þ xNg ð2:3Þ The corresponding ( joint) probability density functions are found by taking the derivatives respectively of Equations (2.1) to (2.3): fXðx1; t1Þ ¼4 @FXðx1; t1Þ @x1 fXðx1; x2; t1; t2Þ ¼4 @2FXðx1; x2; t1; t2Þ @x1@x2 fXðx1; . . . ; xN; t1; . . . ; tNÞ ¼4 @NFXðx1; . . . ; xN; t1; . . . ; tNÞ @x1 Á Á Á @xN ð2:4Þ ð2:5Þ ð2:6Þ Two processes XðtÞ and YðtÞ are called statistically independent if the set of random variables fXðt1Þ; Xðt2Þ; . . . ; XðtNÞg is independent of the set of random variables fYðt10 Þ; Yðt20 Þ; . . . ; YðtM0 Þg, for each arbitrary choice of the time parameters ft1; t2; . . . ; tN; t10 ; t20 ; . . . ; tM0 g. Independence implies that the joint probability density function can be factored in the following way: fX;Y ðx1; . . . ; xN ; y1; . . . ; yM; t1; . . . ; tN ; t10 ; . . . ; tM0 Þ ¼ fXðx1; . . . ; xN; t1; . . . ; tNÞ Á fY ðy1; . . . ; yM; t10 ; . . . ; tM0 Þ ð2:7Þ Thus, the joint probability density function of two independent processes is written as the product of the two marginal probability density functions. 2.1.2 First-Order Stationary Processes A stochastic process is called a first-order stationary process if the first-order probability density function is independent of time. Mathematically this can be stated as fXðx1; t1Þ ¼ fXðx1; t1 þ Þ ð2:8Þ holds for all . As a consequence of this property the mean value of such a process, denoted by XðtÞ, is Z XðtÞ  E½Xðtފ ¼4 x fXðx; tÞ dx ¼ constant ð2:9Þ i.e. it is independent of time. CORRELATION FUNCTIONS 11 2.1.3 Second-Order Stationary Processes A stochastic process is called a second-order stationary process if for the two-dimensional joint probability density function fXðx1; x2; t1; t2Þ ¼ fXðx1; x2; t1 þ ; t2 þ  Þ ð2:10Þ for all . It is easy to verify that Equation (2.10) is only a function of the time difference t2 À t1 and does not depend on the absolute time. In order to gain that insight put  ¼ Àt1. A process that is second-order stationary is first-order stationary as well, since the secondorder joint probability density function uniquely determines the lower-order (in this case first-order) probability density function. 2.1.4 Nth-Order Stationary Processes By extending the reasoning from the last subsection to N random variables Xi ¼ XðtiÞ, for i ¼ 1; . . . ; N, we arrive at an Nth-order stationary process. The Nth-order joint probability density function is once more independent of a time shift; i.e. fXðx1; . . . ; xN ; t1; . . . ; tN Þ ¼ fXðx1; . . . ; xN; t1 þ ; . . . ; tN þ Þ ð2:11Þ for all . A process that is Nth-order stationary is stationary to all orders k N. An Nth-order stationary process where N can have an arbitrary large value is called a strictsense stationary process. 2.2 CORRELATION FUNCTIONS 2.2.1 The Autocorrelation Function, Wide-Sense Stationary Processes and Ergodic Processes The autocorrelation function of a stochastic process is defined as the correlation E½X1 X2Š of the two random variables X1 ¼ Xðt1Þ and X2 ¼ Xðt2Þ. These random variables are achieved by considering all realization values of the stochastic process at the instants of time t1 and t2 (see Figure 2.1). In general it will be a function of these two times instants. The autocorrelation function is denoted as ZZ RXXðt1; t2Þ ¼4 E½Xðt1ÞXðt2ފ ¼ x1x2 fXðx1; x2; t1; t2Þ dx1 dx2 ð2:12Þ Substituting t1 ¼ t and t2 ¼ t1 þ , Equation (2.12) becomes RXXðt; t þ Þ ¼ E½XðtÞ Xðt þ ފ ð2:13Þ 12 STOCHASTIC PROCESSES xn +2(t ) xn +1(t ) xn (t ) xn –1(t ) 0 t X(t 1) X(t 2) Figure 2.1 The autocorrelation of a stochastic process by considering E½Xðt1ÞXðt2ފ Since for a second-order stationary process the two-dimensional joint probability density function depends only on the time difference, the autocorrelation function will also be a function of the time difference . Then Equation (2.13) can be written as RXXðt; t þ  Þ ¼ RXXð Þ ð2:14Þ The mean and autocorrelation function of a stochastic process are often its most characterizing features. Mostly, matters become easier if these two quantities do not depend on absolute time. A second-order stationary process guarantees this independence but at the same time places severe demands on the process. Therefore we define a broader class of stochastic processes, the so-called wide-sense stationary processes. Definition A process XðtÞ is called wide-sense stationary if it satisfies the conditions E½Xðtފ ¼ XðtÞ ¼ constant E½XðtÞ Xðt þ ފ ¼ RXXðÞ ð2:15Þ It will be clear that a second-order stationary process is also wide-sense stationary. The converse, however, is not necessarily true. CORRELATION FUNCTIONS 13 Properties of RXXðsÞ If a process is at least wide-sense stationary then its autocorrelation function exhibits the following properties: 1. jRXXð Þj RXXð0Þ i.e. jRXXðÞj attains its maximum value for  ¼ 0. ð2:16Þ 2. RXXðÀ Þ ¼ RXXð Þ i.e. RXXðÞ is an even function of . ð2:17Þ 3. RXXð0Þ ¼ E½X2ðtފ ð2:18Þ 4. If XðtÞ has no periodic component then RXXðÞ comprises a constant term equal to XðtÞ2, i.e. limjj!1 RXXð Þ ¼ XðtÞ2. 5. If XðtÞ has a periodic component then RXXðÞ will comprise a periodic component as well, and which has the same periodicity. A function that does not satisfy these properties cannot be the autocorrelation function of a wide-sense stationary process. It will be clear from properties 1 and 2 that RXXðÞ is not allowed to exhibit an arbitrary shape. Proofs of the properties: 1. To prove property 1 let us consider the expression E½fXðtÞ Æ Xðt þ Þg2Š ¼ E½X2ðtÞ þ X2ðt þ Þ Æ 2XðtÞ Xðt þ ފ ¼ 2fRXXð0Þ Æ RXXðÞg ! 0 ð2:19Þ Since the expectation E½fXðtÞ Æ Xðt þ Þg2Š is taken over the squared value of a certain random variable, this expectation should be greater than or equal to zero. From the last line of Equation (2.19) property 1 is concluded. 2. The proof of property 2 is quite simple. In the definition of the autocorrelation function substitute t0 ¼ t þ  and the proof proceeds as follows: RXXðÞ ¼ E½XðtÞ Xðt þ ފ ¼ E½Xðt0 À Þ Xðt0ފ ¼ E½Xðt0Þ Xðt0 À  ފ ¼ RXXðÀÞ ð2:20Þ 3. Property 3 follows immediately from the definition of RXXðÞ by inserting  ¼ 0. 4. From a physical point of view most processes have the property that the random variables XðtÞ and Xðt þ Þ are independent when  ! 1. Invoking once more the definition of the 14 STOCHASTIC PROCESSES autocorrelation function it follows that lim  !1 RXX ð Þ ¼ lim  !1 E½XðtÞ Xðt þ  ފ ¼ E½Xðtފ E½Xðt þ ފ ¼ E2½Xðtފ ¼ X2 ð2:21Þ 5. Periodic processes may be decomposed into cosine and sine components according to Fourier analysis. It therefore suffices to consider the autocorrelation function of one such component: E½cosð!t À ÂÞ cosð!t þ ! À Âފ ¼ 1 2 E½cosð! Þ þ cosð2!t þ ! À 2Âފ ð2:22Þ Since our considerations are limited to wide-sense stationary processes, the autocorrelation function should be independent of the absolute time t, and thus the expectation of the last term of the latter expression should be zero. Thus only the term comprising cosð!Þ remains after taking the expectation, which proves property 5. When talking about the mean or expectation (denoted by E½ÁŠ) the statistical average over the ensemble of realizations is meant. Since stochastic processes are time functions we can define another average, namely the time average, given by A½Xðtފ ¼4 lim 1 Z T xðtÞdt T!1 2T ÀT ð2:23Þ When taking this time average only one single sample function can be involved; consequently, expressions like A½Xðtފ and A½XðtÞ Xðt þ ފ will be random variables. Definition A wide-sense stationary process XðtÞ satisfying the two conditions A½Xðtފ ¼ E½Xðtފ ¼ XðtÞ A½XðtÞ Xðt þ ފ ¼ E½XðtÞ Xðt þ ފ ¼ RXXðÞ is called an ergodic process. ð2:24Þ ð2:25Þ In other words, an ergodic process has time averages A½Xðtފ and A½XðtÞ Xðt þ ފ that are non-random because these time averages equal the ensemble averages XðtÞ and RXXðÞ. In the same way as several types of stationary process can be defined, several types of ergodic processes may also be introduced [1]. We will confine ourselves to the forms defined by the Equations (2.24) and (2.25). Ergodicity puts more severe demands on the process than stationarity and it is often hard to prove that indeed a process is ergodic; often it is impossible. In practice ergodicity is often just assumed without proof, unless the opposite is evident. In most cases there is no alternative, as one does not have access to the entire family CORRELATION FUNCTIONS 15 (ensemble) of sample functions, but rather just to one or a few members of it, for example one resistor, transistor or comparable noisy device is available. By assuming ergodicity a number of important statistical properties, such as the mean and the autocorrelation function of a process may be estimated from the observation of a single available realization. Fortunately, it appears that many processes are ergodic, but one should always be aware that at times one can encounter a process that is not ergodic. Later in this chapter we will develop a test for a certain class of ergodic processes. Example 2.1: As an example consider the process XðtÞ ¼ A cosð!t À ÂÞ, with A a constant amplitude, ! a fixed but arbitrary radial frequency and  a random variable that is uniformly distributed on the interval ð0; 2pŠ. The question is whether this process is ergodic in the sense as defined by Equations (2.24) and (2.25). To answer this we determine both the ensemble mean and the time average. For the time average it is found that A½Xðtފ ¼ 1 lim T!1 2T ZT A ÀT cosð!t À ÂÞ dt ¼ 1 lim T!1 2T A 1 ! sinð!t À ÂÞ T ÀT ¼ 0 ð2:26Þ The ensemble mean is Z E½Xðtފ ¼ fÂðÞ A cosð!t À Þ d ¼ 1 2p A Z 0 2 cosð!t À Þ d ¼ 0 ð2:27Þ Hence, time and ensemble averages are equal. Let us now calculate the two autocorrelation functions. For the time-averaged auto- correlation function it is found that A½XðtÞ Xðt þ ފ ¼ lim 1 Z A2 T cosð!t À Þ cosð!t þ ! À Þ dt ¼ T!1 2T 1 lim ÀZT 1 A2 T ½cosð2!t þ ! À 2Þ þ cosð!ފ dt T!1 2T 2 ÀT ð2:28Þ The first term of the latter integral equals 0. The second term of the integrand does not depend on the integration variable. Hence, the autocorrelation function is given by A½XðtÞ Xðt þ  ފ ¼ 1 2 A2 cos ! Next we consider the statistical autocorrelation function E½XðtÞ Xðt þ  ފ ¼ ¼ 1 2p 1 2p Z 2 A2 cosð!t À Þ 0Z 1 A2 2 ½cosð2!t þ 20 cosð!t þ ! ! À 2Þ þ À Þ d cosð! ފ d ð2:29Þ ð2:30Þ 16 STOCHASTIC PROCESSES Of the latter integral the first part is 0. Again, the second term of the integrand does not depend on the integration variable. The autocorrelation function is therefore E½XðtÞ Xðt þ  ފ ¼ 1 2 A2 cos ! ð2:31Þ Hence both first-order means (time average and statistical mean) and second-order means (time-averaged and statistical autocorrelation functions) are equal. It follows that the process is ergodic. The process cosð!t À ÂÞ with fÂðÞ ¼ ðÞ equals the deterministic function cos !t. This process is not ergodic, since it is easily verified that the expectation (in this case the function itself) is time-dependent and thus not stationary, which is a condition for ergodicity. This example, where a probability density function that consists of a  function reduces the process to a deterministic function, has also been mentioned in Chapter 1. & 2.2.2 Cyclo-Stationary Processes A process XðtÞ is called cyclo-stationary (or periodically stationary) if the probability density function is independent of a shift in time over an integer multiple of a constant value T (the period time), so that fXðx1; . . . ; xN ; t1; . . . ; tNÞ ¼ fXðx1; . . . ; xN; t1 þ mT; . . . ; tN þ mTÞ ð2:32Þ for each integer value of m. A cyclo-stationary process is not stationary, since Equation (2.11) is not valid for all values of , but only for discrete values  ¼ mT. However, the discrete-time process XðmT þ Þ is stationary for all values of . A relation exists between cyclo-stationary processes and stationary processes. To see this relation it is evident from Equation (2.32) that FXðx1; . . . ; xN ; t1; . . . ; tNÞ ¼ FXðx1; . . . ; xN ; t1 þ mT; . . . ; tN þ mTÞ ð2:33Þ Next consider the modified process XðtÞ ¼ Xðt À ÂÞ, where XðtÞ is cyclo-stationary and  a random variable that has a uniform probability density function on the period interval ð0; TŠ. Now we define the event A as A ¼ fXðt1 þ  Þ x1; . . . ; XðtN þ Þ xN g ð2:34Þ The probability that this event will occur is PðAÞ ¼ Z 0 T PðAj ¼ Þ fÂðÞ d ¼ 1 T Z 0 T PðAj ¼ Þ d ð2:35Þ For the latter integrand we write PðAj ¼ Þ ¼ PfXðt1 þ  À Þ x1; . . . ; XðtN þ  À Þ xN g ¼ FXðx1; . . . ; xN; t1 þ  À ; . . . ; tN þ  À Þ ð2:36Þ CORRELATION FUNCTIONS 17 Substituting this result in Equation (2.35) yields PðAÞ ¼ 1 T ZT 0 FX ðx1 ; . . . ; xN ; t1 þ  À ; . . . ; tN þ  À Þ d ð2:37Þ As XðtÞ is cyclo-stationary Equation (2.37) is independent of . From Equation (2.34) it follows, therefore, that PðAÞ represents the probability distribution function of the process XðtÞ. Thus we have the following theorem. Theorem 1 If XðtÞ is a cyclo-stationary process with period time T and  is a random variable that is uniformly distributed on the interval ð0; TŠ, then the process XðtÞ ¼ Xðt À ÂÞ is stationary with the probability distribution function FX ðx1 ; .. .; xN ; t1; . .. ; tNÞ ¼ 1 T ZT 0 FX ðx1 ; . . . ; xN ; t1 À ; . . . ; tN À Þ d ð2:38Þ A special case consists of the situation where XðtÞ ¼ pðtÞ is a deterministic, periodic function with period T. Then, as far as the first-order probability distribution function FXðxÞ is concerned, the integral from Equation (2.38) can be interpreted as the relative fraction of time during which XðtÞ is smaller or equal to x. This is easily understood when realizing that for a deterministic function FXðx1; t1Þ is either zero or one, depending on whether pðt1Þ is larger or smaller than x1. If we take XðtÞ ¼ pðtÞ, then this process XðtÞ is strict sense cyclo-stationary and from the foregoing we have the following theorem. Theorem 2 If XðtÞ ¼ pðt À ÂÞ is an arbitrary, periodic waveform with period T and  a random variable that is uniformly distributed on the interval ð0; TŠ, then the process XðtÞ is strictsense stationary and ergodic. The probability distribution function of this process reads FX ðx1 ; . . . ; xN ; t1; . . . ; tN Þ ¼ 1 T Z 0 T Fpðp1; . . . ; pN ; t1 À ; . . . ; tN À Þ d ð2:39Þ The mean value of the process equals E½Xðtފ ¼ Z 1 T pðtÞ dt ¼ A½ pðtފ T0 and the autocorrelation function ð2:40Þ RXXð Þ ¼ 1 T Z 0 T pðtÞ pðt þ Þ dt ¼ A½ pðtÞ pðt þ  ފ ð2:41Þ 18 STOCHASTIC PROCESSES This latter theorem is a powerful expedient when proving strict-sense stationarity and ergodicity of processes that often occur in practice. In such cases the probability distribution function is found by means of the integral given by Equation (2.39). For this integral the same interpretation is valid as for that from Equation (2.38). From the probability distribution function the probability density function can be derived using Equation (2.4). By adding a random phase  , with  uniformly distributed on the interval ð0; TŠ, to a cyclostationary process the process can be made stationary. Although this seems to be an artificial operation, it is not so from a physical point of view. If we imagine that the ensemble of realizations originates from a set of signal generators, let us say sinusoidal wave generators, all of them tuned to the same frequency, then the waves produced by these generators will as a rule not be synchronized in phase. Example 2.2: The process XðtÞ ¼ cosð!tÞ is not stationary, its mean value being cosð!tÞ; however, it is cyclo-stationary. On the contrary, the modified process XðtÞ ¼ cosð!t À ÂÞ, with  uniformly distributed on the interval ð0; 2pŠ, is strict-sense stationary and ergodic, based on Theorem 2. The ergodicity of this process was already concluded when dealing with Example 2.1. Moreover, we derived the autocorrelation function of this process as 1 2 cosð! Þ. Let us now elaborate this example. We will derive the probability distribution and density functions, based on Theorem 2. For this purpose remember that the probability distribution function is given by Equation (2.39) and that this integral is interpreted as the relative fraction of time during which XðtÞ is smaller than or equal to x. This interpretation has been further explained by means of Figure 2.2. In this figure one complete period of a cosine is presented. The constant value x is indicated. The duration that the given realization is smaller than or equal to x has been drawn by means of the bold line pieces, which are indicated by T1 and T2, and the complete second half of the cycle, which has a length of p. It can be seen that the line pieces T1 and T2 are of equal length, namely arcsin x. Finally, the probability distribution function is found by adding all the bold line pieces and dividing the result by the period 2p. This leads to the probability distribution function FXðxÞ ¼ PfXðtÞ 8 >< 1 2 þ 1  arcsin x; xg ¼ >: 0; 1; jxj 1 x < À1 x>1 ð2:42Þ x T1 T2 π ωt Figure 2.2 Figure to help determine the probability distribution function of the random phased cosine Fx (x ) 1 CORRELATION FUNCTIONS 19 f x(x ) ½ 1 π −1 0 (a) 1x −1 0 (b) 1x Figure 2.3 (a) The probability distribution function and (b) the probability density function of the random phased cosine This function is depicted in Figure 2.3(a). From the probability distribution function the probability density function is easily derived by taking the derivative, i.e. fX ðxÞ ¼4 dFX ðxÞ ¼  1 p pffiffi1ffiffiffiffiffi 1Àx2 ; dx 0; jxj 1 jxj > 1 ð2:43Þ This function has been plotted in Figure 2.3(b). Note the asymptotic values of the function for both x ¼ 1 and x ¼ À1. & Example 2.3: The random data signal X XðtÞ ¼ Anpðt À nTÞ n ð2:44Þ with An a stationary sequence of binary random variables that are selected out of the set fÀ1; þ1g and with autocorrelation sequence E½An AkŠ ¼ E½An AnþmŠ ¼ E½An AnÀmŠ ¼ Rm ð2:45Þ constitutes a cyclo-stationary process, where Theorems 1 and 2 can be applied. Properties of this random data signal will be derived in more detail later on (see Section 4.5). & 2.2.3 The Cross-Correlation Function The cross-correlation function of two stochastic processes XðtÞ and YðtÞ is defined as RXY ðt; t þ  Þ ¼4 E½XðtÞ Yðt þ ފ ð2:46Þ 20 STOCHASTIC PROCESSES XðtÞ and YðtÞ are jointly wide-sense stationary if both XðtÞ and YðtÞ are wide-sense stationary and if the cross-correlation function RXY ðt; t þ Þ is independent of the absolute time parameter, i.e. RXY ðt; t þ  Þ ¼ E½XðtÞ Yðt þ ފ ¼ RXY ðÞ ð2:47Þ Properties of RXYðsÞ If two processes XðtÞ and YðtÞ are jointly wide-sense stationary, then the crosscorrelation function has the following properties: 1. RXY ðÀÞ ¼ RYXð Þ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2. jRXY ðÞj RXXð0Þ RYY ð0Þ 3. jRXY ðÞj 1 2 ½RXX ð0Þ þ RYY ð0ފ ð2:48Þ ð2:49Þ ð2:50Þ A function that does not satisfy these properties cannot be the cross-correlation function of two jointly wide-sense stationary processes. Proofs of the properties: 1. Property 1 is proved as follows. In the definition of the cross-correlation function replace t À  by t0 and do some manipulation as shown below: RXY ðÀÞ ¼ E½XðtÞ Yðt À ފ ¼ E½Xðt0 þ Þ Yðt0ފ ¼ E½Yðt0Þ Xðt0 þ  ފ ¼ RYXð Þ ð2:51Þ 2. To prove property 2 we consider the expectation of the process fXðtÞ þ cYðt þ Þg2, where c is a constant; i.e. we investigate E½fXðtÞ þ cYðt þ Þg2Š ¼ E½X2ðtÞ þ c2Y2ðt þ Þ þ 2cXðtÞ Yðt þ ފ ¼ E½X2ðtފ þ c2E½Y2ðt þ ފ þ 2cE½XðtÞ Yðt þ ފ ¼ c2RYY ð0Þ þ 2cRXY ðÞ þ RXXð0Þ ð2:52Þ This latter expression is a quadratic form as a function of c, and since it is the expectation of a quantity squared, this expression can never be less than zero. As a consequence the discriminant cannot be positive; i.e. R2XY ð Þ À RXXð0Þ RYY ð0Þ 0 From this property 2 follows. ð2:53Þ CORRELATION FUNCTIONS 21 3. Property 3 is a consequence of the well-known fact that the arithmetic mean of two positive numbers is always greater than or equal to their geometric mean. If for two processes XðtÞ and YðtÞ RXY ðt; t þ Þ ¼ 0; for all t and  ð2:54Þ then we say that XðtÞ and YðtÞ are orthogonal processes. In case two processes XðtÞ and YðtÞ are statistically independent, the cross-correlation function can be written as RXY ðt; t þ Þ ¼ E½Xðtފ E½Yðt þ ފ ð2:55Þ If, moreover, XðtÞ and YðtÞ are at least wide-sense stationary, then Equation (2.55) becomes RXY ðt; t þ  Þ ¼ X Y ð2:56Þ Two stochastic processes XðtÞ and YðtÞ are called jointly ergodic if the individual processes are ergodic and if the time-averaged cross-correlation function equals the statistical crosscorrelation function, i.e. if A½XðtÞ Yðt þ ފ ¼ E½XðtÞ Yðt þ ފ ¼ RXY ðÞ ð2:57Þ In practice one more often uses spectra, to be dealt with in the next chapter, than correlation functions, as the measurement equipment for spectra is more developed than that for correlations. In that chapter it will be shown that the correlation function acts as the basis for calculating the spectrum. However, the correlation function in itself also has interesting applications, as is concluded from the following examples. Example 2.4: It will be shown that based on a correlation function, by means of the system described in this example, one is able to measure a distance. Consider a system (see Figure 2.4) where a signal source produces a random signal, being a realization of a stochastic process. Let us Reflecting object Source Transmitter waves X (t ) Y (t ) Receiver Correlator RXY (τ) Figure 2.4 Set-up for measuring a distance based on the correlation function 22 STOCHASTIC PROCESSES RXX (τ) 1 RXX (τ) α 0 τ (a) 0 τ=T τ (b) Figure 2.5 (a) The autocorrelation function of the transmitted signal and (b) the measured crosscorrelation function of the distance measuring set-up assume the process to be wide-sense stationary. The signal is applied to a transmitter that produces a wave in a transmission medium; let it be an acoustic wave or an electromagnetic wave. We denote the transmitted random wave by XðtÞ. Let us further suppose that the transmitted wave strikes a distant object and that this object (partly) reflects the wave. Then this reflected wave will travel backwards to the position of the measuring equipment. The measuring equipment comprises a receiver and the received signal is denoted as YðtÞ. Both the transmitted signal XðtÞ and the received signal YðtÞ are applied to a correlator that produces the cross-correlation function RXY ðÞ. In the next section it will be explained how this correlation equipment operates. The reflected wave will be a delayed and attenuated version of the transmitted wave; i.e. we assume YðtÞ ¼ Xðt À TÞ, where T is the total travel time. The cross-correlation function will be RXY ðÞ ¼ E½XðtÞ Yðt þ ފ ¼ E½XðtÞ Xðt À T þ ފ ¼ RXXð À TÞ ð2:58Þ Most autocorrelation functions have a peak at  ¼ 0, as shown in Figure 2.5(a). Let us normalize this peak to unity; then the cross-correlation result will be as depicted in Figure 2.5(b). From this latter picture a few conclusions may be drawn with respect to the application at hand. Firstly, when we detect the position of the peak in the crosscorrelation function we will be able to establish T and if the speed of propagation of the wave in the medium is known, then the distance of the object can be derived from that. Secondly, the relative height of the peak can be interpreted as a measure for the size of the object. It will be clear that this method is very useful in such ranging systems as radar and underwater acoustic distance measurement. Most ranging systems use pulsed continuous wave (CW) signals for that. The advantage of the system presented here is the fact that for the transmitted signal a noise waveform is used. Such a waveform cannot easily be detected by the probed object, in contrast to the pulsed CW systems, since it has no replica available of the transmitted signal and therefore is not able to perform the correlation. The probed object only observes an increase in received noise level. & CORRELATION FUNCTIONS 23 Example 2.5: Yet another interesting example of the application of the correlation concept is in the field of reducing the distortion of received information signals. Let us suppose that a private subscriber has on the roof of his house an antenna for receiving TV broadcast signals. Due to a tall building near his house the TV signal is reflected, so that the subscriber receives the signal from a certain transmitter twice, once directly from the transmitter and a second time reflected from the neighbouring building. On the TV screen this produces a ghost of the original picture and spoils the picture. We call the direct signal XðtÞ and the reflected one will then be Xðt À TÞ, where T represents the difference in travel time between the direct and the reflected signal. The total received signal is therefore written as YðtÞ ¼ XðtÞ þ Xðt À TÞ. Let us consider the autocorrelation function of this process: RYY ðÞ ¼ E½fXðtÞ þ Xðt À TÞg fXðt þ Þ þ Xðt À T þ ÞgŠ ¼ E½XðtÞ Xðt þ Þ þ XðtÞ Xðt À T þ Þ þ Xðt À TÞ Xðt þ Þ þ 2Xðt À TÞ Xðt À T þ ފ ¼ ð1 þ 2ÞRXXðÞ þ RXXð À TÞ þ RXXð þ TÞ ð2:59Þ The autocorrelation function RYY ðÞ of the received signal YðtÞ will consist of that of the original signal RXXðÞ multiplied by 1 þ 2 and besides that two shifted versions of RXXðÞ. These versions are multiplied by and shifted in time over respectively T and ÀT. If it is assumed that the autocorrelation function RXXðÞ shows the same peaked appearance as in the previous example, then the autocorrelation function of the received signal YðtÞ looks like that in Figure 2.6. Let us once again suppose that both and T can be determined from this measurement. Then we will show that these parameters can be used to reduce the distortion caused by the reflection from the nearby building; namely we delay the received signal by an amount of T and multiply this delayed version by . This delayed and multiplied version is subtracted from the received signal, so that after this operation we have the signal ZðtÞ ¼ YðtÞ À Yðt À TÞ. Inserting the undistorted signal XðtÞ into this yields ZðtÞ ¼ YðtÞ À Yðt À TÞ ¼ XðtÞ þ Xðt À TÞ À Xðt À TÞ À 2Xðt À 2TÞ ¼ XðtÞ À 2Xðt À 2TÞ ð2:60Þ RYY (τ) 1+α2 α α T 0 T τ Figure 2.6 The autocorrelation function of a random signal plus its delayed and attenuated versions 24 STOCHASTIC PROCESSES From this equation it is concluded that indeed the term with a delay of T has been removed. One may argue that, instead, the term 2Xðt À 2TÞ has been introduced. That is right, but it is not unreasonable to assume that the reflection coefficient is (much) less than unity, so that this newly introduced term is smaller by the factor of compared to the distortion in the received signal. If this is nevertheless unacceptable then a further reduction is achieved by also adding the term 2Yðt À 2TÞ to the received signal. This removes the distortion at 2T and in its turn introduces a term that is still smaller by an amount of 3 at a delay of 3T, etc. In this way the distortion may be reduced to an arbitrary small amount. & Apart from these examples there are several applications that use correlation as the basic signal processing method for extracting information from an observation. 2.2.4 Measuring Correlation Functions In a practical situation it is impossible to measure a correlation function. This is due to the fact that we will never have available the entire ensemble of sample functions of the process in question. Even if we did have them then it would nevertheless be impossible to cope with an infinite number of sample functions. Thus we have to confine ourselves to a limited class of processes, e.g. to the class of ergodic processes. We have established before that most of the time it is difficult or even impossible to determine whether a process is ergodic or not. Unless the opposite is clear, we will assume ergodicity in practice; this greatly simplifies matters, especially for measuring correlation functions. This assumption enables the wanted correlation function based on just a single sample function to be determined, as is evident from Equation (2.25). In Figure 2.7 a block schematic is shown for a possible set-up to measure a crosscorrelation function RXY ðÞ, where the assumption has to be made that the processes XðtÞ and YðtÞ are jointly ergodic. The sample functions xðtÞ and yðtÞ should be applied to the inputs at least starting at t ¼ ÀT þ  up until t ¼ T þ . The signal xðtÞ is delayed and applied to a multiplier whereas yðtÞ is applied undelayed to a second input of the same multiplier. The multiplier’s output is applied to an integrator that integrates over a period 2T. Looking at this scheme we conclude that the measured output is Roð; TÞ ¼ 1 2T Z Tþ ÀT þ xðt À Þ yðtÞ dt ¼ 1 2T ZT ÀT xðtÞ yðt þ  Þ dt ð2:61Þ If the integration time 2T is taken long enough, and remembering the assumption on the jointly ergodicity of XðtÞ and YðtÞ, the measured value Roð; TÞ will approximate RXY ðÞ. By x (t ) delay τ y (t ) 1 T+τ 2T dt −T+τ Ro (τ,T ) Figure 2.7 Measurement scheme for correlation functions CORRELATION FUNCTIONS 25 varying  the function can be measured for different values of the argument. By simply short-circuiting the two inputs and applying a single signal xðtÞ to this common input, the autocorrelation function RXXðÞ is measured. In practice only finite measuring times can be realized. In general this will introduce an error in the measured result. In the next example this point will be further elaborated. Example 2.6: Let us consider the example that has been subject of our studies several times before, namely the cosine waveform with amplitude A and random phase that has a uniform distribution over one period of the cosine. This process has been described in Example 2.1. Suppose we want to measure the autocorrelation function of this process using the set-up given in Figure 2.7. The inputs are short-circuited and the signal is applied to these common inputs. If the given process is substituted in Equation (2.61), we find Roð; TÞ ¼ ¼ 1 2T A2 ZT A2 cosð!t Z ÀT T ½cos ! þ À Þ cosð!t þ ! À cosð2!t þ ! À 2ފ Þ dt dt 4T ÀT ð2:62Þ In this equation the random variable  has not been used, but the specific value  that corresponds to the selected realization of the process. The first term in the integrand of this integral produces ðA2=2Þ cosð!Þ, the value of the autocorrelation function of this process, as was concluded in Example 2.1. The second term in the integrand must be a measurement error. The magnitude of this error is determined by evaluating the corresponding integral. This yields eð; TÞ ¼ A2 2 cosð! À 2Þ sinð2!T Þ 2!T ð2:63Þ The error has an oscillating character as a function of T, while the absolute value of the error decreases inversely with T. At large values of T the error approaches 0. If, for example, the autocorrelation function has to be measured with an accuracy of 1%, then the condition 1=ð2!TÞ < 0:01 should be fulfilled, or equivalently the measurement time should satisfy 2T > 100=!. Although this analysis looks nice, its applicability is limited. In practice the autocorrelation function is not known beforehand; that is why we want to measure it. Thus the above error analysis cannot be carried out. The solution to this problem consists of doing a besteffort measurement and then to make an estimate of the error in the correlation function. Looking back, it possible to decide whether the measurement time was long enough for the required accuracy. If not, the measurement can be redone using a larger (estimated) measurement time based on the error analysis. In this way accuracy can be iteratively improved. & 26 STOCHASTIC PROCESSES 2.2.5 Covariance Functions The concept of covariance of two random variables can be extended to stochastic processes. The autocovariance function of a stochastic process is defined as CXXðt; t þ Þ ¼4 E½fXðtÞ À E½Xðtފg fXðt þ Þ À E½Xðt þ ފgŠ ð2:64Þ This can be written as CXXðt; t þ Þ ¼ RXXðt; t þ Þ À E½Xðtފ E½Xðt þ ފ The cross-covariance function of two processes XðtÞ and YðtÞ is defined as ð2:65Þ CXY ðt; t þ Þ ¼4 E½fXðtÞ À E½Xðtފg fYðt þ Þ À E½Yðt þ ފgŠ ð2:66Þ or CXY ðt; t þ Þ ¼ RXY ðt; t þ Þ À E½Xðtފ E½Yðt þ ފ ð2:67Þ For processes that are at least jointly wide-sense stationary the second expressions in the right-hand sides of Equations (2.65) and (2.67) can be simplified, yielding CXXð Þ ¼ RXXð Þ À X2 ð2:68Þ and CXY ðÞ ¼ RXY ðÞ À X Y ð2:69Þ respectively. From Equation (2.68) and property 4 of the autocorrelation function in Section 2.2.1 it follows immediately that lim CXXð Þ ¼ 0 j j!1 ð2:70Þ provided the process XðtÞ does not have a periodic component. If in Equation (2.64) the value  ¼ 0 is used we obtain the variance of the process. In the case of wide-sense stationary processes the variance is independent of time, and using Equation (2.68) we arrive at 2X ¼4 E½fXðtÞ À E½Xðtފg2Š ¼ CXXð0Þ ¼ RXXð0Þ À X2 ð2:71Þ If for two processes CXY ðt; t þ  Þ  0 ð2:72Þ GAUSSIAN PROCESSES 27 then these processes are called uncorrelated processes. According to Equation (2.67) this has as a consequence RXY ðt; t þ Þ ¼ E½Xðtފ E½Yðt þ ފ ð2:73Þ Since this latter equation is identical to Equation (2.55), it follows that independent processes are uncorrelated. The converse is not necessarily true, unless the processes are jointly Gaussian processes (see Section 2.3). 2.2.6 Physical Interpretation of Process Parameters In the previous sections stochastic processes have been described from a mathematical point of view. In practice we want to relate these descriptions to physical concepts such as a signal, represented, for example, by a voltage or a current. In these cases the following physical interpretations are connected to the parameters of the stochastic processes:  The mean XðtÞ is proportional to the d.c. component of the signal.  The squared mean value XðtÞ2 is proportional to the power in the d.c. component of the signal.  The mean squared value X2ðtÞ is proportional to the total average power of the signal.  The variance 2X ¼4 X2ðtÞ À XðtÞ2 is proportional to the power in the time-varying components of the signal, i.e. the a.c. power.  The standard deviation X is the square root of the mean squared value of the timevarying components, i.e. the root-mean-square (r.m.s.) value. In Chapter 6 the proportionality factors will be deduced. Now it suffices to say that this proportionality factor becomes unity in case the load is purely resistive and equal to one. Although the above interpretations serve to make the engineer familiar with the practical value of stochastic processes, it must be stressed that they only apply to the special case of signals that can be modelled as ergodic processes. 2.3 GAUSSIAN PROCESSES Several processes can be modelled by what is called a Gaussian process; among these is the thermal noise process that will be presented in Chapter 6. As the name suggests, these processes are described by Gaussian distributions. Recall that the probability density function of a Gaussian random variable X is defined by [1–5] " # fX ðxÞ ¼ p1 ffiffiffiffiffi X 2p exp À ðx À XÞ2 22X ð2:74Þ 28 STOCHASTIC PROCESSES The Gaussian distribution is frequently encountered in engineering and science. When considering two jointly Gaussian random variables X and Y we sometimes need the joint probability density function, as will become apparent in the sequel ( " #) fXY ðx; yÞ ¼ 1pffiffiffiffiffiffiffiffiffiffiffiffiffi 2pXY 1 À 2 exp À1 2ð1 À 2Þ ðx À XÞ2 2X À 2ðx À XÞðy X Y À YÞ þ ðy À YÞ2 2Y ð2:75Þ where  is the correlation coefficient defined by  ¼4 E½ðX À XÞðY À Yފ X Y ð2:76Þ For N jointly Gaussian random variables X1, X2; . . . ; XN, the joint probability density function reads " # fX1 X2 ÁÁÁXN ðx1; x2; . . . ; xN Þ ¼ jCÀX 1 j1 2 ð2pÞN=2 exp À ðx À XÞTCÀX 1ðx À XÞ 2 ð2:77Þ where we define the vector 2 3 x1 À X1 x À X ¼4 6664 x2 À ... X2 7775 xN À XN ð2:78Þ and the covariance matrix 2 3 C11 C12 Á Á Á C1N CX ¼4 6664 C21 ... C22 ... ÁÁÁ C2N ... 7775 CN1 CN2 Á Á Á CNN ð2:79Þ In the foregoing we used xT for the matrix transpose, CÀ1 for the matrix inverse and jCj for the determinant. The elements of the covariance matrix are defined by Cij ¼4 EðXi À XiÞðXj À Xjފ ð2:80Þ The diagonal elements of the covariance matrix equal the variances of the various random variables, i.e. Cii ¼ 2Xi . It is easily verified that Equations (2.74) and (2.75) are special cases of Equation (2.77). GAUSSIAN PROCESSES 29 Gaussian variables as described above have a few interesting properties, which have their consequences for Gaussian processes. These properties are [1–5]: 1. Gaussian random variables are completely specified only by their first and second order moments, i.e. by their means, variances and covariances. This is immediately apparent, since these are the only quantities present in Equation (2.77). 2. When Gaussian random variables are uncorrelated, they are independent. For uncorrelated random variables (i.e.  ¼ 0) the covariance matrix is reduced to a diagonal matrix. It is easily verified from Equation (2.77) that in such a case the probability density function of N variables can be written as the product of N functions of the type given in Equation (2.74). 3. A linear combination of Gaussian random variables produces another Gaussian variable. For the proof of this see reference [2] and Problem 8.3. We are now able to define a Gaussian stochastic process. Referring to Equation (2.77), a process XðtÞ is called a Gaussian process if the random variables X1 ¼ Xðt1Þ, X2 ¼ Xðt2Þ; . . . ; XN ¼ XðtNÞ are jointly Gaussian and thus satisfy " # fX ðx1 ; . . . ; xN ; t1; . . . ; tN Þ ¼ jCÀX 1 j1 2 ð2pÞN=2 exp À ðx À XÞTCÀX 1ðx À XÞ 2 ð2:81Þ for all arbitrary N and for any set of times t1; . . . ; tN. Now the mean values Xi of XðtiÞ are Xi ¼ E½Xðtiފ ð2:82Þ and the elements of the covariance matrix are Cij ¼ E½ðXi À XiÞðXj À Xjފ ¼ E½fXðtiÞ À E½XðtiފgfXðtjÞ À E½Xðtjފg ¼ CXXðti; tjÞ ð2:83Þ which is the autocovariance function as defined by Equation (2.64). Gaussian processes have a few interesting properties. Properties of Gaussian Processes 1. Gaussian processes are completely specified by their mean E½Xðtފ and autocorrelation function RXXðti; tjÞ. 2. A wide-sense stationary Gaussian process is also strict-sense stationary. 3. If the jointly Gaussian processes XðtÞ and YðtÞ are uncorrelated, then they are independent. 4. If the Gaussian process XðtÞ is passed through a linear time-invariant system, then the corresponding output process YðtÞ is also a Gaussian process. 30 STOCHASTIC PROCESSES These properties are closely related to the properties of jointly Gaussian random variables previously discussed in this section. Let us briefly comment on the properties: 1. We saw before that the joint probability density function is completely determined when the mean and autocovariance are known. However, these two quantities as functions of time in their turn determine the autocorrelation function (see Equation (2.68)). 2. The nth-order probability density function of a Gaussian process only depends on the two functions E½Xðtފ and CXXðt; t þ Þ. When the process is wide-sense stationary then these functions do not depend on the absolute time t, and as a consequence fXðx1; . . . ; xN ; t1; . . . ; tNÞ ¼ fXðx1; . . . ; xN ; t1 þ ; . . . ; tN þ  Þ ð2:84Þ Since this is valid for all arbitrary N and all , it is concluded that the process is strictsense stationary. 3. This property is a straightforward consequence of the property of jointly random variables discussed before. 4. Passing a process through a linear time-invariant system is described by a convolution, which may be considered as the limit of a weighted sum of samples of the input process. From the preceding we know that a linear combination of Gaussian variables produces another Gaussian variable. 2.4 COMPLEX PROCESSES A complex stochastic process is defined by ZðtÞ ¼4 XðtÞ þ jYðtÞ ð2:85Þ with XðtÞ and YðtÞ real stochastic processes. Such a process is said to be stationary if XðtÞ and YðtÞ are jointly stationary. Expectation and the autocorrelation function of a complex stochastic process are defined as E½Zðtފ ¼4 E½XðtÞ þ jYðtފ ¼ E½Xðtފ þ jE½Yðtފ ð2:86Þ and RZZ ðt; t þ Þ ¼4 E½ZÃðtÞ Zðt þ ފ ð2:87Þ where à indicates the complex conjugate. For the autocovariance function the definition of Equation (2.87) is used, where ZðtÞ is replaced by the stochastic process ZðtÞ À E½Zðtފ. This yields CZZðt; t þ  Þ ¼ RZZðt; t þ  Þ À EýZðtފ E½Zðt þ  ފ ð2:88Þ DISCRETE-TIME PROCESSES 31 The cross-correlation function of two complex processes ZiðtÞ and ZjðtÞ reads RZiZj ðt; t þ  Þ ¼ E½ZiÃðtÞ Zjðt þ ފ ð2:89Þ and the cross-covariance function is found from Equation (2.89) by replacing Zi; jðtÞ with Zi; jðtÞ À E½Zi; jðtފ; this yields CZiZj ðt; t þ  Þ ¼ RZiZj ðt; t þ Þ À EýZiðtފ E½Zjðt þ ފ ð2:90Þ In the chapters that follow we will work exclusively with real processes, unless it is explicitly indicated that complex processes are considered. One may wonder why the correlation functions of complex processes are defined in the way it has been done in Equations (2.87) and (2.89). The explanation for this arises from an engineering point of view; namely the given expressions of the correlation functions evaluated for  ¼ 0 have to result in the expectation of the squared process for real processes. In engineering calculations real processes are replaced many times by complex processes of the form I ¼ ^I expðj!tÞ (for a current) or V ¼ V^ expðj!tÞ (for a voltage). In these cases the correlation function for  ¼ 0 should be a quantity that is proportional to the mean power. The given definitions satisfy this requirement. 2.5 DISCRETE-TIME PROCESSES In Chapter 1 the discrete-time process was introduced by sampling a continuous stochastic process. However, at this point we are not yet able to develop a sampling theorem for stochastic processes analogously to that for deterministic signals [1]. We will derive such a theorem in Chapter 3. This means that in this section we deal with random sequences as such, irrespective of their origin. In Chapter 1 we introduced the notation X½nŠ for random sequences. In this section we will assume that the sequences are real. However, they can be complex valued. Extension to complex discrete-time processes is similar to what was derived in the former section. In the next subsection we will resume the most important properties of discrete-time processes. Since such processes are actually special cases of continuous stochastic processes the properties are self-evident. 2.5.1 Mean, Correlation Functions and Covariance Functions The mean value of a discrete-time process is found by Z1 E½X½nŠŠ ¼ X½nŠ ¼4 x fXðx; nÞ dx À1 ð2:91Þ Recall that the process is time-discrete but the x values are continuous, so that indeed the expectation (or ensemble mean) is written as an integral over a continuous probability density function. This function describes the random variable X½nŠ by considering all 32 STOCHASTIC PROCESSES ... xn +2[n] xn +1[n] xn [n] xn –1[n] 0 X [n1] n X [n2] Figure 2.8 Random variables X½n1Š and X½n2Š that arise when considering the ensemble values of the discrete-time process X½nŠ at fixed positions n1 and n2 possible ensemble realizations of the process at a fixed integer position for example n1 (see Figure 2.8). For real processes the autocorrelation sequence is defined as ZZ RXX½n1; n2Š ¼4 E½X½n1Š X½n2ŠŠ ¼4 x1x2 fXðx1; x2; n1; n2Þ dx1 dx2 ð2:92Þ where the process is now considered at two positions n1 and n2 jointly (see again Figure 2.8). For the autocovariance sequence of this process (compare to Equation (2.65)) CXX½n1; n2Š ¼ RXX½n1; n2Š À E½X½n1ŠŠ E½X½n2ŠŠ ð2:93Þ The cross-correlation and cross-covariance sequences are defined analogously, namely respectively as RXY ½n; n þ mŠ ¼4 E½X½nŠ Y½n þ mŠŠ ð2:94Þ and (compare with Equation (2.67)) CXY ½n; n þ mŠ ¼4 RXY ½n; n þ mŠ À E½X½nŠŠ E½Y½n þ mŠŠ ð2:95Þ A discrete-time process is called wide-sense stationary if the next two conditions hold jointly: E½X½nŠŠ ¼ constant RXX½n; n þ mŠ ¼ RXX½mŠ ð2:96Þ ð2:97Þ i.e. the autocorrelation sequence only depends on the difference m of the integer positions. SUMMARY 33 Two discrete-time processes are jointly wide-sense stationary if they are individually wide-sense stationary and moreover RXY ½n; n þ mŠ ¼ RXY ½mŠ ð2:98Þ i.e. the cross-correlation sequence only depends on the difference m of the integer positions. The time average of a discrete-time process is defined as A½X½nŠŠ ¼4 lim N!1 1 2N þ 1 XN n¼ÀN X½nŠ ð2:99Þ A wide-sense stationary discrete-time process is ergodic if the two conditions are satisfied A½X½nŠŠ ¼ E½X½nŠŠ ¼ X½nŠ ð2:100Þ and A½X½nŠ X½n þ mŠŠ ¼ E½X½nŠ X½n þ mŠŠ ¼ RXX½mŠ ð2:101Þ 2.6 SUMMARY An ensemble is the set of all possible realizations of a stochastic process XðtÞ. A realization or sample function is provided by a random selection out of this ensemble. For the description of stochastic processes a parameter is added to the well-known definitions of the probability distribution function and the probability density function, namely the time parameter. This means that these functions in the case of a stochastic process are as a rule functions of time. When considering stationary processes certain time dependencies disappear; we thus arrive at first-order and second-order stationary processes, which are useful for practical applications. The correlation concept is in random signal theory, analogously to probability theory, defined as the expectation of the product of two random variables. For the autocorrelation function these variables are XðtÞ and Xðt þ Þ, while for the cross-correlation function of two processes the quantities XðtÞ and Yðt þ Þ are used in the definition. A wide-sense stationary process is a process where the mean value is constant and the autocorrelation function only depends on , not on the absolute time t. When calculating the expectations the time t is considered as a parameter; i.e. in these calculations t is given a fixed value. The random variable is the variable based on which outcome of the realization is chosen from the ensemble. When talking about ‘mean’ we have in mind the ensemble mean, unless it is explicitly indicated that a different definition is used (for instance the time average). In the case of an ergodic process the first- and second-order time averages equal the first- and second-order ensemble means, respectively. The theorem that has been presented on cyclo-stationary processes plays an important role in ‘making stationary’ certain classes of processes. The covariance functions of stochastic processes are the correlation functions of these processes minus their own process mean values. Physical interpretations of several stochastic concepts have been presented. Gaussian processes get special attention as they are of practical importance and possess a few 34 STOCHASTIC PROCESSES interesting and convenient properties. Complex processes are defined analogously to the usual method for complex variables. Finally, several definitions and properties of continuous stochastic processes are redefined for discrete-time processes. 2.7 PROBLEMS 2.1 All sample functions of a stochastic process are constant, i.e. XðtÞ ¼ C ¼ constant, where C is a discrete random variable that may assume the values C1 ¼ 1, C2 ¼ 3 and C3 ¼ 4, with probabilities of 0.5, 0.3 and 0.2, respectively. (a) Determine the probability density function of XðtÞ. (b) Calculate the mean and variance of XðtÞ. 2.2 Consider a stationary Gaussian process with a mean of zero. (a) Determine and sketch the probability density function of this process after passing it through an ideal half-wave rectifier. (b) Same question for the situation where the process is applied to a full-wave rectifier. 2.3 A stochastic process comprises four sample functions, namely xðt; s1Þ ¼ 1, xðt; s2Þ ¼ t, xðt; s3Þ ¼ cos t and xðt; s4Þ ¼ 2 sin t, which occur with equal probabilities. (a) Determine the probability density function of XðtÞ. (b) Is the process stationary in any sense? 2.4 Consider the process XN XðtÞ ¼ An cosð!ntÞ þ Bn sinð!ntÞ n¼1 where An and Bn are random variables that are mutually uncorrelated, have zero mean and of which E½A2nŠ ¼ E½B2nŠ ¼ 2 The quantities f!ng are constants. (a) Calculate the autocorrelation function of XðtÞ. (b) Is the process wide-sense stationary? 2.5 Consider the stochastic process XðtÞ ¼ A cosð!0tÞ þ B sinð!0tÞ, with !0 a constant and A and B random variables. What are the conditions for A and B in order for XðtÞ to be wide-sense stationary? 2.6 Consider the process XðtÞ ¼ A cosð!0t À ÂÞ, where A and  are independent random variables and  is uniformly distributed on the interval ð0; 2pŠ. (a) Is this process wide-sense stationary? (b) Is it ergodic? 2.7 Consider the two processes PROBLEMS 35 XðtÞ ¼ A cosð!0tÞ þ B sinð!0tÞ YðtÞ ¼ A cosð!0tÞ À B sinð!0tÞ with A and B independent random variables, both with zero mean and equal variance of 2. The angular frequency !0 is constant. (a) Are the processes XðtÞ and YðtÞ wide-sense stationary? (b) Are they jointly wide-sense stationary? 2.8 Consider the stochastic process XðtÞ ¼ A sinð!0t À ÂÞ, with A and !0 constants, and  a random variable that is uniformly distributed on the interval ð0; 2pŠ. We define a new process by means of YðtÞ ¼ X2ðtÞ. (a) Are XðtÞ and YðtÞ wide-sense stationary? (b) Calculate the autocorrelation function of YðtÞ. (c) Calculate the cross-correlation function of XðtÞ and YðtÞ. (d) Are XðtÞ and YðtÞ jointly wide-sense stationary? (e) Calculate and sketch the probability distribution function of YðtÞ. (f) Calculate and sketch the probability density function of YðtÞ. 2.9 Repeat Problem 2.8 when XðtÞ is half-wave rectified. Use Matlab to plot the autocorrelation function. 2.10 Repeat Problem 2.8 when XðtÞ is full-wave rectified. Use Matlab to plot the autocorrelation function. 2.11 The function pðtÞ is defined as pðtÞ ¼  1; 0 t 3 4 T 0; all other values of t By means of this function we define the stochastic process X 1 XðtÞ ¼ pðt À nT À ÂÞ n¼À1 where  is a random variable that is uniformly distributed on the interval ½0; TÞ. (a) Sketch a possible realization of XðtÞ. (b) Calculate the mean value of XðtÞ. (c) Calculate and sketch the autocorrelation function of XðtÞ. (d) Calculate and sketch the probability distribution function of XðtÞ. (e) Calculate and sketch the probability density function of XðtÞ. (f) Calculate the variance of XðtÞ. 36 STOCHASTIC PROCESSES 2.12 Two functions p1ðtÞ and p2ðtÞ are defined as  1; p1ðtÞ ¼ 0; 0 t 1 3 T all other values of t and p2ðtÞ ¼  1; 0; 0 t 2 3 T all other values of t Based on these functions the stochastic processes XðtÞ and YðtÞ are defined as X 1 XðtÞ ¼ p1ðt À nT À ÂÞ n¼À1 X 1 YðtÞ ¼ p2ðt À nT À ÂÞ n¼À1 and WðtÞ ¼4 XðtÞ þ YðtÞ where  is a random variable that is uniformly distributed on the interval ½0; TÞ. (a) Sketch possible realizations of XðtÞ and YðtÞ. (b) Calculate and sketch the autocorrelation function of XðtÞ. (c) Calculate and sketch the autocorrelation function of YðtÞ. (d) Calculate and sketch the autocorrelation function of WðtÞ. (e) Calculate the power in WðtÞ, i.e. E½W2ðtފ. 2.13 The processes XðtÞ and YðtÞ are independent with a mean value of zero and autocorrelation functions RXXðÞ ¼ expðÀjjÞ and RYY ðÞ ¼ cosð2pÞ, respectively. (a) Derive the autocorrelation function of the sum W1ðtÞ ¼ XðtÞ þ YðtÞ. (b) Derive the autocorrelation function of the difference W2ðtÞ ¼ XðtÞ À YðtÞ. (c) Calculate the cross-correlation function of W1ðtÞ and W2ðtÞ. 2.14 In Figure 2.9 the autocorrelation function of a wide-sense stationary stochastic process XðtÞ is given. (a) Calculate the value of E½Xðtފ. (b) Calculate the value of E½X2ðtފ. (c) Calculate the value of 2X. 2.15 Starting from the wide-sense stationary process XðtÞ we define a new process as YðtÞ ¼ XðtÞ À Xðt þ TÞ. (a) Show that the mean value of YðtÞ is zero, even if the mean value of XðtÞ is not zero. (b) Show that 2Y ¼ 2fRXXð0Þ À RXXðTÞg. RXX (τ) 25 PROBLEMS 37 9 –5 0 5 τ Figure 2.9 (c) If YðtÞ ¼ XðtÞ þ Xðt þ TÞ find expressions for E½Yðtފ and 2Y . Compare these results with the answers found in (a) and (b). 2.16 Determine for each of the following functions whether it can be the autocorrelation function of a real wide-sense stationary process XðtÞ. (a) RXXðÞ ¼ uðÞ expðÀÞ. (b) RXXðÞ ¼ 3 sinð7Þ. (c) RXXðÞ ¼ ð1 þ 2ÞÀ1. (d) RXXðÞ ¼ À cosð2Þ expðÀjjÞ. (e) RXXðÞ ¼ 3½sinð4Þ=ð4ފ2. (f ) RXXðÞ ¼ 1 þ 3 sinð8Þ=ð8Þ. 2.17 Consider the two processes XðtÞ and YðtÞ. Find expressions for the autocorrelation function of WðtÞ ¼ XðtÞ þ YðtÞ in the case where: (a) XðtÞ and YðtÞ are correlated; (b) XðtÞ and YðtÞ are uncorrelated; (c) XðtÞ and YðtÞ are uncorrelated and have mean values of zero. 2.18 The voltage of the output of a noise generator is measured using a d.c. voltmeter and a true root-mean-square (r.m.s.) meter that has a series capacitor at its input. The noise is known to be Gaussian and stationary. The reading of the d.c. meter is 3 V and that of the r.m.s. meter is 2 V. Derive an expression for the probability density function of the noise and make a plot of it using Matlab. 2.19 Two real jointly wide-sense stationary processes XðtÞ and YðtÞ are used to define two complex processes as follows: Z1ðtÞ ¼ XðtÞ þ jYðtÞ and Z2ðtÞ ¼ Xðt À TÞ À jYðt À TÞ Calculate the cross-correlation function of the processes Z1ðtÞ and Z2ðtÞ. 38 STOCHASTIC PROCESSES 2.20 A voltage source is described as V ¼ 5 cosð!0t À ÂÞ, where  is a random variable that is uniformly distributed on ½0; 2pÞ. This source is applied to an electric circuit and as a consequence the current flowing through the circuit is given by I ¼ 2 cosð!0t À  þ p=6Þ. (a) Calculate the cross-correlation function of V and I. (b) Calculate the electrical power that is absorbed by the circuit. (c) If in general an harmonic voltage at the terminals of a circuit is described by its complex notation V ¼ V^ exp½ jð!t À Âފ and the corresponding current that is flowing into the circuit by a similar notation I ¼ ^I exp½ jð!t À  þ ފ, with  a constant, show that the electrical power absorbed by the circuit is written as Pel ¼ ðV^ ^I cos Þ=2. 2.21 Consider a discrete-time wide-sense stationary process X½nŠ. Show that for such a process 3RXX½0Š ! j4RXX½1Š þ 2RXX½2Šj 3 Spectra of Stochastic Processes In Chapter 2 stochastic processes have been considered in the time domain exclusively; i.e. we used such concepts as the autocorrelation function, the cross-correlation function and the covariance function to describe the processes. When dealing with deterministic signals, we have the frequency domain at our disposal as a means to an alternative, dual description. One may wonder whether for stochastic processes a similar duality exists. This question is answered in the affirmative, but the relationship between time domain and frequency domain descriptions is different compared to deterministic signals. Hopping from one domain to the other is facilitated by the well-known Fourier transform and its inverse transform. A complicating factor is that for a random waveform (a sample function of the stochastic process) the Fourier transform generally does not exist. 3.1 THE POWER SPECTRUM Due to the problems with the Fourier transform, a theoretical description of stochastic processes must basically start in the time domain, as given in Chapter 2. In this chapter we will confine ourselves exclusively to wide-sense stationary processes with the autocorrelation function RXXðÞ. Let us assume that it is allowed to apply the Fourier transform to RXX ð Þ. Theorem 3 The Wiener–Khinchin relations are Z1 SXXð!Þ ¼ RXXð Þ expðÀj! Þ d À1 RXX ð Þ ¼ 1 2p Z1 À1 SXX ð!Þ expðj! Þ d! ð3:1Þ ð3:2Þ Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 40 SPECTRA OF STOCHASTIC PROCESSES SXX (ω) −ω0 ω0 ω −ω0−dω ω0+dω Figure 3.1 Interpretation of SXXð!Þ The function SXXð!Þ has an interesting interpretation, as will follow from the sequel. For that purpose we put the variable  equal to zero in Equation (3.2). This yields RXX ð0Þ ¼ E½X2ðtފ ¼ 1 2p Z1 À1 SXX ð!Þ d! ð3:3Þ However, from Equation (2.18) it is concluded that RXXð0Þ equals the mean squared value of the process; this is called the mean power of the process, or just the power of the process. Now it follows from Equation (3.3) that SXXð!Þ represents the way in which the total power of the process is spread over the different frequency components. This is clear since integrating SXXð!Þ over the entire frequency axis produces the total power of the process. In other words, 2SXXð!0Þd!=ð2pÞ is the power at the output of the bandpass filter with the passband transfer function  Hð!Þ ¼ 1; !0 < j!j < !0 þ d! 0; elsewhere ð3:4Þ when the input of this filter consists of the process XðtÞ. This is further explained by Figure 3.1. Due to this interpretation the function SXXð!Þ is called the power spectral density, or briefly the power spectrum of the process XðtÞ. The Wiener–Khinchin relations state that the autocorrelation function and the power spectrum of a wide-sense stationary process are a Fourier transform pair. From the given interpretation the properties of the Fourier transform are as follows. Properties of SXXð!Þ 1. SXXð!Þ ! 0 2. SXXðÀ!Þ ¼ SXXð!Þ; for a real process XðtÞ 3. ImfSXXð!Þg  0 where ImfÁg is defined as the imaginary part of the quantity between the braces 4. 1 2p R1 À1 SXX ð!Þ d! ¼ E½X2ðtފ ¼ RXX ð0Þ ¼ PXX ð3:5Þ ð3:6Þ ð3:7Þ ð3:8Þ THE POWER SPECTRUM 41 Proofs of the properties: 1. Property 1 is connected to the interpretation of SXXð!Þ and a detailed proof will be given in Chapter 4. 2. Property 2 states that the power spectrum is an even function of !. The proof of this property is based on Fourier theory and the fact that for a real process the autocorrelation function RXXðÞ is real and even. The proof proceeds as follows: Z1 SXXð!Þ ¼ RXXð Þ½cosð!Þ À j sinð! ފ d À1 ð3:9Þ Since RXXðÞ is real and even, the product of this function and a sine is odd. Therefore, this product makes no contribution to the integral, which runs over a symmetrical range of the integration variable. The remaining part is a product of RXXðÞ and a cosine, both being even, resulting in an even function of !. 3. The third property, SXXð!Þ being real, is proved as follows. Let us define the complex process XðtÞ ¼ RðtÞ þ jIðtÞ, where RðtÞ and IðtÞ are real processes and represent the real and imaginary part of XðtÞ, respectively. Then after some straightforward calculations the autocorrelation function of XðtÞ is RXXðÞ ¼ RRRðÞ þ RIIð Þ þ j½RRIðÞ À RIRðފ ð3:10Þ Inserting this into the Fourier integral produces the power spectrum Z1 SXXð!Þ ¼ ½RRRð Þ þ RIIðފ½cosð! Þ À j sinð! ފ À1 þ ½RRIðÞ À RIRðފ½ j cosð! Þ þ sinð!ފ d ð3:11Þ The product of the sum of the two autocorrelation functions and the sine gives an odd result and consequently does not contribute to the integral. Using Equation (2.48), the difference RRIðÞ À RIRðÞ can be rewritten as RRIðÞ À RRIðÀÞ. This is an odd function and multiplied by a cosine the result remains odd. Thus, this product does not contribute to the integral either. Since all imaginary parts cancel out on integration, the resulting power spectrum will be real. 4. Property 4 follows immediately from the definition of Equation (3.2) and the definition of the autocorrelation function (see Equation (2.18)). Example 3.1: Consider once more the stochastic process XðtÞ ¼ A cosð!0t À ÂÞ, with A and !0 constants and  a random variable that is uniformly distributed on the interval ð0; 2Š. We know that this process is often met in practice. The autocorrelation function of this process has been 42 SPECTRA OF STOCHASTIC PROCESSES SXX (ω) −ω0 0 ω0 ω Figure 3.2 The power spectrum of a random phased cosine shown to be RXXð Þ ¼ 1 2 A2 cosð!0 Þ (see Example 2.1). From a table of Fourier transforms (see Appendix G) it is easily revealed that SXX ð!Þ ¼ p 2 A2½ð! À !0Þ þ ð! þ !0ފ ð3:12Þ This spectrum has been depicted in Figure 3.2 and consists of two  functions, one at ! ¼ !0 and another one at ! ¼ À!0. Since the phase is random, introducing an extra constant phase to the cosine does not have any effect on the result. Thus, instead of the cosine a sine wave could also have been taken. & Example 3.2: The second example is also important from a practical point of view, namely the spectrum of an oscillator. From physical considerations the process can be written as XðtÞ ¼ ÉA ðctoÞs½¼!0RtÀtþ1 Éðtފ, with A and !0 NðÞd, where NðtÞ is constants and ÉðtÞ a random walk process defined a so-called white noise process; i.e. the spectrum of N by ðtÞ has a constant value for all frequencies. It can be shown that the autocorrelation function of the process XðtÞ is [8] RXX ð Þ ¼ A2 2 expðÀ j jÞ cosð!0 Þ ð3:13Þ where !0 is the nominal angular frequency of the oscillator and the exponential is due to random phase fluctuations. This autocorrelation function is shown in Figure 3.3(a). It will be clear that A is determined by the total power of the oscillator and from the Fourier table (see Appendix G) the power spectrum SXX ð!Þ ¼ 2 A2=2 þ ð! À !0Þ2 þ 2 A2=2 þ ð! þ !0Þ2 ð3:14Þ follows. This spectrum has been depicted in Figure 3.3(b) and is called a Lorentz profile. & THE BANDWIDTH OF A STOCHASTIC PROCESS 43 RXX (τ) SXX (ω) τ −ω0 ω0 ω (a) (b) Figure 3.3 (a) The autocorrelation function and (b) the power spectrum of an oscillator 3.2 THE BANDWIDTH OF A STOCHASTIC PROCESS The r.m.s. bandwidth We of a stochastic process is defined using the second normalized moment of the power spectrum, i.e. We2 ¼4 RÀR1À111!S2XSXXXð!ð!Þ Þd!d! ð3:15Þ This definition is, in its present form, only used for lowpass processes, i.e. processes where SXXð!Þ has a significant value at ! ¼ 0 and at low frequencies, and decreasing values of SXXð!Þ at increasing frequency. Example 3.3: In this example we will calculate the r.m.s. bandwidth of a very simple power spectrum, namely an ideal lowpass spectrum defined by  SXXð!Þ ¼ 1; 0; for j!j < B for j!j ! B ð3:16Þ Inserting this into the definition of Equation (3.15) yields We2 ¼ RÀRBÀBBB!2d!d! ¼ 1 B2 3 ð3:17Þ pffiffi The r.m.s. bandwidth is in this case We ¼ B= 3. This bandwidth might have been expected to be equal to B; the difference is explained by the quadratic weight in the numerator with respect to frequency. & 44 SPECTRA OF STOCHASTIC PROCESSES In case of bandpass processes (see Subsection 4.4.1) the second, central, normalized moment is used in the definition We2 ¼4 4 R01ðR!01ÀS!XX0ðÞ!2SÞXdX!ð!Þ d! ð3:18Þ where the mean frequency !0 is defined by !0 ¼4 RR0101!SSXXXXðð!!ÞÞdd!! ð3:19Þ the first normalized moment of SXXð!Þ. A bandpass process is a process where the power spectral density function is confined around a frequency !"0 and which has a negligible value (zero or almost zero) at ! ¼ 0. The necessity of the factor of 4 in Equation (3.18) compared to Equation (3.15) is explained by the next example. Example 3.4: In this example we will consider the r.m.s. bandwidth of an ideal bandpass process with the power spectrum ( SXXð!Þ ¼ 1; for j! À !0j < B 2 and j! þ !0j < B 2 0; elsewhere ð3:20Þ The r.m.s. bandwidth follows from the definition of Equation (3.18): We2 ¼ 4 R!!00ÀþRBB!=!=2020ÀþðB!B==2À2 d!!0Þ2 d! ¼ 1 3 B2 ð3:21Þ pffiffi which reveals that the r.m.s. bandwidth equals We ¼ B= 3. Both the spectrum of the ideal lowpass process from Example 3.3 and the spectrum of the ideal bandpass process from this example are presented in Figure 3.4. From a physical point of view the two processes should SXX (ω) 1 SXX (ω) B B 1 −B 0 (a) Bω −ω0 −B/2−ω0−ω0 +B/2 0 (b) ω0−B/2 ω0 ω0+B/2 ω Figure 3.4 (a) Power spectrum of the ideal lowpass process and (b) the power spectrum of the ideal bandpass process THE CROSS-POWER SPECTRUM 45 have the same bandwidth and indeed both Equations (3.17) and (3.21) have the same outcome. This is only the case if the factor of 4 is present in Equation (3.18). & 3.3 THE CROSS-POWER SPECTRUM Analogous to the preceding section, we can define the cross-power spectral density function, or briefly the cross-power spectrum, as the Fourier transform of the cross-correlation function Z1 SXY ð!Þ ¼ RXY ðÞ expðÀj! Þ d À1 ð3:22Þ with the corresponding inverse transform RXY ð Þ ¼ 1 2p Z1 À1 SXY ð!Þ expðj! Þ d! ð3:23Þ It can be seen that the processes XðtÞ and YðtÞ have to be jointly wide-sense stationary. A physical interpretation of this spectrum cannot always be given. The function SXY ð!Þ often acts as an auxiliary quantity in a few specific problems, such as in bandpass processes (see Section 5.2). Moreover, it plays a role when two (or even more) signals are added. Let us consider the process ZðtÞ ¼ XðtÞ þ YðtÞ; then the autocorrelation is RZZ ð Þ ¼ RXXð Þ þ RYY ð Þ þ RXY ð Þ þ RYXðÞ ð3:24Þ From this latter equation the total power of ZðtÞ is PXX þ PYY þ PXY þ PYX and it follows that the process ZðtÞ contains, in general, more power than the sum of the powers of the individual signals. This apparently originates from the correlation of the signals. The cross- power spectra show how the additional power components PXY and PYX are spread over the different frequencies, namely PXY ¼4 1 2p Z1 À1 SXY ð!Þ d! ð3:25Þ and PYX ¼4 1 2p Z1 À1 SYX ð!Þ d! ð3:26Þ The total amount of additional power may play an important role in situations where an information-carrying signal has to be processed in the midst of additive noise or interference. Moreover, the cross-power spectrum is used to describe bandpass processes (see Chapter 5). From Equation (3.24) it will be clear that the power of ZðtÞ equals the sum of the powers in XðtÞ and YðtÞ if the processes XðtÞ and YðtÞ are orthogonal. 46 SPECTRA OF STOCHASTIC PROCESSES Properties of SXYðxÞ for real processes 1. SXY ð!Þ ¼ SYXðÀ!Þ ¼ SÃYXð!Þ 2. RefSXY ð!Þg and RefSYXð!Þg are even functions of ! ImfSXY ð!Þg and ImfSYXð!Þg are odd functions of ! where RefÁg and ImfÁg are the real and imaginary parts, respectively, of the quantity in the braces 3. If XðtÞ and YðtÞ are independent, then SXY ð!Þ ¼ SYXðÀ!Þ ¼ 2X Yð!Þ 4. If XðtÞ and YðtÞ are orthogonal, then SXY ð!Þ  SYXð!Þ  0 5. If XðtÞ and YðtÞ are uncorrelated, then ð3:27Þ ð3:28Þ ð3:29Þ ð3:30Þ ð3:31Þ SXY ð!Þ ¼ SYXðÀ!Þ ¼ 2pX Yð!Þ ð3:32Þ Proofs of the properties: 1. Property 1 is proved by invoking Equation (2.48) and the definition of the cross-power spectrum Z1 Z1 SXY ð!Þ ¼ RXY ð Þ expðÀj! Þ d ¼ RXY ðÀÞ expðj!Þ d À1 À1 Z1 ¼ RYXð Þ expðj! Þ d ¼ SYXðÀ!Þ À1 ð3:33Þ and from this latter line it follows also that SYXðÀ!Þ ¼ SÃYXð!Þ. 2. In contrast to SXXð!Þ the cross-power spectrum will in general be a complex-valued function. Property 2 follows immediately from the definition Z1 Z1 SXY ð!Þ ¼ RXY ðÞ cosð! Þ d À j RXY ð Þ sinð! Þ d À1 À1 ð3:34Þ For real processes the cross-correlation function is real as well and the first integral represents the real part of the power spectrum and is obviously even. The second integral represents the imaginary part of the power spectrum which is obviously odd. 3. From Equation (2.56) it is concluded that in this case RXY ðÞ ¼ X Y and its Fourier transform equals the right-hand member of Equation (3.30). MODULATION OF STOCHASTIC PROCESSES 47 4. The fourth property is quite straightforward. For orthogonal processes, by definition, RXY ðÞ ¼ RYXðÞ  0, and so are the corresponding Fourier transforms. 5. In the given situation, from Equations (2.69) and (2.72) it is concluded that RXY ðÞ ¼ X Y. Fourier transform theory says that the transform of a constant is a  function of the form given by Equation (3.32). 3.4 MODULATION OF STOCHASTIC PROCESSES In many applications (such as in telecommunications) a situation is often met where signals are modulated and synchronously demodulated. In those situations the signal is applied to a multiplier circuit, while a second input of the multiplier is a harmonic signal (sine or cosine waveform), called the carrier (see Figure 3.5). We will analyse the spectrum of the output process YðtÞ when the spectrum of the input process XðtÞ is known. In doing so we will assume that the cosine function of the carrier has a constant frequency !0, but a random phase  that is uniformly distributed on the interval (0,2] and is independent of XðtÞ. The output process is then written as YðtÞ ¼ XðtÞA0 cosð!0t À ÂÞ ð3:35Þ The amplitude A0 of the carrier is supposed to be constant. The autocorrelation function of the output YðtÞ is found by applying the definition to this latter expression, yielding RYY ðt; t þ  Þ ¼ A20E½XðtÞ cosð!0t À ÂÞXðt þ  Þ cosð!0t þ !0 À Âފ ð3:36Þ At the start of this chapter we stated that we will confine our analysis to wide-sense stationary processes. We will invoke this restriction for the input process XðtÞ; however, this does not guarantee that the output process YðtÞ is also wide-sense stationary. Therefore we used the notation RYY ðt; t þ Þ in Equation (3.36) and not RYY ðÞ. Elaborating Equation (3.36) yields RYY ðt; t þ  Þ ¼ ¼ A20 2 A20 2 RXX ð RXX ð Þ E½cosð2!0t þ !0  Þ 1 2p Z 0 2p cosð2!0t À 2ÂÞ þ cos !0Š þ !0 À 2Þ d þ cos  !0 ¼ A20 2 RXX ð Þ cos !0 ð3:37Þ X (t ) SXX (ω) Y (t ) SYY (ω) A 0cos(ω0t- Θ) Figure 3.5 A product modulator or mixer 48 SPECTRA OF STOCHASTIC PROCESSES SXX (ω) 1 0 ω (a) SYY (ω) A 2 0 4 –ω0 0 (b) ω0 ω Figure 3.6 The spectra at (a) input ðSXXð!ÞÞ and (b) output ðSYY ð!ÞÞ of a product modulator From Equation (3.37) it is seen that RYY ðt; t þ Þ is independent of t. The mean value of the output is calculated using Equation (3.35); since  and XðtÞ have been assumed to be independent the mean equals the product of the mean values E½Xðtފ and E½cosð!0t À Âފ. From Example 2.1 we know that this latter mean value is zero. Thus it is concluded that the output process YðtÞ is wide-sense stationary, since its autocorrelation function is independent of t and so is its mean. Transforming Equation (3.37) to the frequency domain, we arrive at our final result: SYY ð!Þ ¼ A20 4 ½SXX ð! À !0Þ þ SXX ð! þ !0ފ ð3:38Þ In Figure 3.6 an example has been sketched of a spectrum SXXð!Þ. Moreover, the corresponding spectrum SYY ð!Þ as it appears at the output of the product modulator is presented. In this figure it has been assumed that XðtÞ is a lowpass process. The analysis can, however, be applied in a similar way to processes with a different character, e.g. bandpass processes. As a consequence of the modulation we observe a shift of the baseband spectrum to the carrier frequency !0 and a shift to À!0; actually besides a shift there is also a split-up. This is analogous to the modulation of deterministic signals, a difference being that when dealing with deterministic signals we use the signal spectrum, whereas when dealing with stochastic processes we have to use the power spectrum. Example 3.5: The method of modulation is in practice used for demodulation as well; demodulation in this way is called synchronous or coherent demodulation. The basic idea is that multiplication of MODULATION OF STOCHASTIC PROCESSES 49 SYY (ω) –ω0 0 ω0 ω (a) SZZ (ω) –2ω0 0 (b) 2ω0 ω Figure 3.7 (a) The spectra of a modulated signal and (b) the output of the corresponding signal after synchronous demodulation by a product modulator a signal with an harmonic wave (sine or cosine) shifts the power spectrum by an amount equal to the frequency of the harmonic signal. This shift is twofold: once to the right and once to the left over the frequency axis. Let us apply this procedure to the spectrum of the modulated signal as given in Figure 3.6(b). This figure has been redrawn in Figure 3.7(a). When this spectrum is both shifted to the right and to the left and added, the result is given by Figure 3.7(b). The power spectrum of the demodulated signal consists of three parts: 1. A copy of the original spectrum about À2!0; 2. A copy of the original spectrum about 2!0; 3. Two copies about ! ¼ 0. The first two copies may be removed by a lowpass filter, whereas the copies around zero actually represent the recovered baseband signal from Figure 3.6(a). & Besides modulation and demodulation, multiplication may also be applied for frequency conversion. Modulation and demodulation are therefore examples of frequency conversion (or frequency translation) that can be achieved by using multipliers. 3.4.1 Modulation by a Random Carrier In certain systems stochastic processes are used as the carrier for modulation. An example is pseudo noise sequences that are used in CDMA (Code Division Multiple Access) systems [9]. The spectrum of such pseudo noise sequences is much wider than that of the modulating 50 SPECTRA OF STOCHASTIC PROCESSES signal. A second example is a lightwave communication system, where sources like light emitting diodes (LEDs) also have a bandwidth much wider than the modulating signal. These wideband sources can be described as stochastic processes and we shall denote them by ZðtÞ. For the modulation we use the scheme of Figure 3.5, where the sinusoidal carrier is replaced by this ZðtÞ. If the modulating process is given by XðtÞ, then the modulation signal at the output reads YðtÞ ¼ XðtÞ ZðtÞ ð3:39Þ Assuming that both processes are wide-sense stationary, the autocorrelation function of the output is written as RYY ðt; t þ Þ ¼ E½XðtÞZðtÞ Xðt þ ÞZðt þ ފ ¼ E½XðtÞXðt þ Þ ZðtÞZðt þ ފ ð3:40Þ It is reasonable to assume that the processes XðtÞ and ZðtÞ are independent. Then it follows that RYY ðt; t þ Þ ¼ RXXðÞ RZZðÞ ð3:41Þ which means that the output is wide-sense stationary as well. Transforming Equation (3.41) to the frequency domain produces the power spectrum of the modulation SYY ð!Þ ¼ 1 2p SXX ð!Þ Ã SZZ ð!Þ ð3:42Þ where à presents the convolution operation. When ZðtÞ has a bandpass characteristic and XðtÞ is a baseband signal, then the modulated signal will be shifted to the bandpass frequency range of the noise-like carrier signal. It is well known that convolution exactly adds the spectral extent of the spectra of the individual signals when they are strictly band-limited. In the case of signals with unlimited spectral extent, the above relation holds approximately in terms of bandwidths [7]. This means that, for example, in the case of CDMA the spectrum of the transmitted signal is much wider than that of the information signal. Therefore, this modulation is also called the spread spectrum technique. On reception, a synchronized version of the pseudo noise signal is generated and synchronous demodulation recovers the information signal. De-spreading is therefore performed in the receiver. 3.5 SAMPLING AND ANALOGUE-TO-DIGITAL CONVERSION In modern systems extensive use is made of digital signal processors (DSPs), due to the fact that these processors can be programmed and in this way can have a flexible functionality. Moreover, the speed is increasing to such high values that the devices become suitable for many practical applications. However, most signals to be processed are still analogue, such as signals from sensors and communication systems. Therefore sampling of the analogue signal is needed prior to analogue-to-digital (A/D) conversion. In this section we will consider both the sampling process and A/D conversion. SAMPLING AND ANALOGUE-TO-DIGITAL CONVERSION 51 3.5.1 Sampling Theorems First we will recall the well-known sampling theorem for deterministic signals [7,10], since we need it to describe a sampling theorem for stochastic processes. Theorem 4 Suppose that the deterministic signal f ðtÞ has a band-limited Fourier transform Fð!Þ; i.e. Fð!Þ ¼ 0 for j!j > W. Then the signal can exactly be recovered from its samples, if the samples are taken at a sampling rate of at least 1=Ts, where Ts ¼ p W ð3:43Þ The reconstruction of f ðtÞ from its samples is given by f ðtÞ ¼ nX ¼1 n¼À1 f ðnTsÞ sin Wðt À nTsÞ Wðt À nTsÞ ð3:44Þ The minimum sampling frequency 1=Ts ¼ W=p ¼ 2F is called the Nyquist frequency, where F ¼ W=ð2pÞ is the maximum signal frequency component corresponding to the maximum angular frequency W. The sampling theorem is understood by considering ideal sampling of the signal. Ideal sampling is mathematically described by multiplying the continuous signal f ðtÞ by an equidistant sequence of  pulses as was mentioned in Chapter 1. This multiplication is equivalent to a convolution in the frequency domain. From Appendix G it is seen that an infinite sequence of  pulses in the time domain is in the frequency domain an infinite sequence of  pulses as well. A sampling rate of 1=Ts in the time domain gives a distance of 2p=Ts between adjacent  pulses in the frequency domain. This means that in the frequency domain the spectrum Fð!Þ of the signal is reproduced infinitely many times shifted over n2p=Ts, with n an integer running from À1 to 1. This is further explained by means of Figure 3.8. From this figure the reconstruction and minimum sampling rate is also understood; namely the original spectrum Fð!Þ, and thus the signal f ðtÞ, is recovered from the ideal lowpass filter F (ω) –W W 2π/Ts ω Figure 3.8 The spectrum of a sampled signal and its reconstruction 52 SPECTRA OF STOCHASTIC PROCESSES periodic spectrum by applying ideal lowpass filtering to it. However, ideal lowpass filtering in the time domain is described by a sinc function, as given in Equation (3.44). This sinc function provides the exact interpolation in the time domain. When the sampling rate is increased, the different replicas of Fð!Þ become further apart and this will still allow lowpass filtering to filter out Fð!Þ, as is indicated in Figure 3.9(a). However, decreasing the sampling rate below the Nyquist rate introduces overlap of the replicas (see Figure 3.9(b)) and thus distortion; i.e. the original signal f ðtÞ can no longer be exactly recovered from its samples. This distortion is called aliasing distortion and is depicted in Figure 3.9(c). For stochastic processes we can formulate a similar theorem. Theorem 5 Suppose that the wide-sense stationary process XðtÞ has a band-limited power spectrum SXXð!Þ; i.e. SXXð!Þ ¼ 0 for j!j > W. Then the process can be recovered from its samples in the mean-squared error sense, if the samples are taken at a sampling rate of at least 1=Ts, where Ts ¼ p W ð3:45Þ The reconstruction of XðtÞ from its samples is given by X^ ðtÞ ¼ nX ¼1 n¼À1 XðnTsÞ sin Wðt À nTsÞ Wðt À nTsÞ ð3:46Þ The reconstructed process X^ðtÞ converges to the original process XðtÞ in the mean- squared error sense, i.e. E½fX^ðtÞ À XðtÞg2Š ¼ 0 ð3:47Þ Proof: We start the proof by remarking that the autocorrelation function RXXðtÞ is a deterministic function with a band-limited Fourier transform SXXð!Þ, according to the conditions mentioned in Theorem 5. As a consequence, Theorem 4 may be applied to it. The reconstruction of RXXðtÞ from its samples is written as RXX ðtÞ ¼ nX ¼1 n¼À1 RXX ðnTsÞ sin Wðt À nTsÞ Wðt À nTsÞ ¼ nX ¼1 n¼À1 RXX ðnTs Þ sinc½W ðt À nTsފ ð3:48Þ For the proof we need two expressions that are derived from Equation (3.48). The first one is nX ¼1 RXXðtÞ ¼ RXXðnTs À T1Þ sinc½Wðt À nTs þ T1ފ n¼À1 ð3:49Þ SAMPLING AND ANALOGUE-TO-DIGITAL CONVERSION 53 ideal lowpass filter F (ω) π Ts ω (a) F (ω) −W W ω (b) ideal lowpass filter π Ts ω (c) Figure 3.9 The sampled signal spectrum (a) when the sampling rate is higher than the Nyquist rate; (b) when it is lower than the Nyquist rate; (c) aliasing distortion This expression follows from the fact that the sampling theorem only prescribes a minimum sampling rate, not the exact positions of the samples. The reconstruction is independent of shifting all samples over a certain amount. Another relation we need is nX ¼1 RXXðt À T2Þ ¼ RXXðnTsÞ sinc½Wðt À T2 À nTsފ n¼À1 nX ¼1 ¼ RXXðnTs À T2Þ sinc½Wðt À nTsފ n¼À1 ð3:50Þ The second line above follows by applying Equation (3.49) to the first line. The mean-squared error between the original process and its reconstruction is written as E½fX^ðtÞ À XðtÞg2Š ¼ E½X^2ðtÞ þ X2ðtÞ À 2X^ðtÞXðtފ ¼ E½X^2ðtފ þ RXXð0Þ À 2E½X^ðtÞXðtފ ð3:51Þ 54 SPECTRA OF STOCHASTIC PROCESSES The first term of this latter expression is elaborated as follows: " # nX ¼1 mX ¼1 E½X^2ðtފ ¼ E XðnTsÞ sinc½Wðt À nTsފ XðmTsÞ sinc½Wðt À mTsފ n¼À1 m¼À1 nX ¼1 mX ¼1 ¼ E½XðnTsÞXðmTsފ sinc½Wðt À nTsފ sinc½Wðt À mTsފ n¼À1(m¼À1 ) nX ¼1 mX ¼1 ¼ RXXðmTs À nTsÞ sinc½Wðt À mTsފ sinc½Wðt À nTsފ n¼À1 m¼À1 ð3:52Þ To the expression in braces we apply Equation (3.50) with T2 ¼ nTs. This yields nX ¼1 E½X^2ðtފ ¼ RXXðt À nTsÞ sinc½Wðt À nTsފ ¼ RXXð0Þ n¼À1 ð3:53Þ This equality is achieved when we once more invoke Equation (3.50), but now with T2 ¼ t. In the last term of Equation (3.51) we insert Equation (3.46). This yields " # nX ¼1 E½XðtÞX^ðtފ ¼ E XðtÞ XðnTsÞ sinc½Wðt À nTsފ n¼À1 nX ¼1 ¼ E½XðtÞ XðnTsފ sinc½Wðt À nTsފ n¼À1 nX ¼1 ¼ RXXðt À nTsÞ sinc½Wðt À nTsފ ¼ RXXð0Þ n¼À1 ð3:54Þ This result follows from Equation (3.53). Inserting Equations (3.53) and (3.54) into Equation (3.51) yields E½fX^ðtÞ À XðtÞg2Š ¼ 0 ð3:55Þ This completes the proof of the theorem. By means of applying the sampling theorem, continuous stochastic processes can be converted to discrete-time processes, without any information getting lost. This facilitates the processing of continuous processes by DSPs. 3.5.2 A/D Conversion For processing in a computer or DSP the discrete-time process has to be converted to a discrete random sequence. That conversion is called analogue-to-digital (A/D) conversion. In this conversion the continuous sample values have to be converted to a finite set of discrete values; this is called quantization. It is important to realize that this is a crucial step, since this final set is an approximation of the analogue (continuous) samples. As in all SAMPLING AND ANALOGUE-TO-DIGITAL CONVERSION 55 output A ∆ input –A Figure 3.10 Quantization characteristic approximations there are differences, called errors, between the original signal and the converted one. These errors cannot be restored in the digital-to-analogue reconversion. Thus, it is important to carefully consider the errors. For the sake of better understanding we will assume that the sample values do not exceed both certain positive and negative values, let us say jxj A. Furthermore, we set the number of possible quantization levels equal to L þ 1. The conversion is performed by a quantizer that has a transfer function as given in Figure 3.10. This quantization characteristic is presented by the solid staircase shaped line, whereas on the dashed line the output is equal to the input. According to this characteristic the quantizer rounds off the input to the closest of the output levels. We assume that the quantizer accommodates the dynamic range of the signal, which covers the range of fÀA; Ag. The difference between the two lines represents the quantization error eq. The mean squared value of this error is calculated as follows. The difference between two successive output levels is denoted by Á. The exact distribution of the signal between A and ÀA is, as a rule, not known, but let us make the reasonable assumption that the analogue input values are uniformly distributed between two adjacent levels for all stages. Then the value of the probability density function of the error is 1=Á and runs from ÀÁ=2 to Á=2. The mean value of the error is then zero and the variance e2 ¼ Z 1 Á=2 Á ÀÁ=2 e2qdeq ¼ Á2 12 ð3:56Þ It is concluded that the power of the error is proportional to the square of the quantization step size Á. This means that this error can be reduced to an acceptable value by selecting an appropriate number of quantization levels. This error introduces noise in the quantization 56 SPECTRA OF STOCHASTIC PROCESSES process. By experience it has been established that the power spectral density of the quantization noise extends over a larger bandwidth than the signal bandwidth [11]. Therefore, it behaves approximately as white noise (see Chapter 6). Example 3.6: As an example of quantization we consider a sinusoidal signal of amplitude A, half the value of the range of the quantizer. In that case this range is fully exploited and no overload will occur. Recall that the number of output levels was L þ 1, so that the step size is Á ¼ 2A L ð3:57Þ Consequently, the power of the quantization error is Pe ¼ e2 ¼ A2 3L2 ð3:58Þ Remembering that the power in a sinusoidal wave with amplitude A amounts to A2=2, the ratio of the signal power to the quantization noise power follows. This ratio is called the signal-to-quantization noise ratio and is  S ¼ 3L2 Nq 2 ð3:59Þ When expressing this quantity in decibels (see Appendix B) it becomes  S 10 log ¼ 1:8 þ 20 log L dB Nq ð3:60Þ Using binary words of length n to present the different quantization levels, the number of these levels is 2n. Inserting this into Equation (3.60) it is found that  S 10 log % 1:8 þ 6n dB Nq ð3:61Þ for large values of L. Therefore, when adding one bit to the word length, the signal-toquantization noise ratio increases by an amount of 6 dB. For example, the audio CD system uses 16 bits, which yields a signal-to-quantization noise ratio of 98 dB, quite an impressive value. & The quantizer characterized by Figure 3.10 is a so-called uniform quantizer, i.e. all steps have the same size. It will be clear that the signal-to-quantization noise ratio can be substantially smaller than the value given by Equation (3.60) if the input amplitude is much smaller than A. This is understood when realizing that in that case the signal power goes down but the quantization noise remains the same. SPECTRUM OF DISCRETE-TIME PROCESSES 57 Non-uniform quantizers have a small step size for small input signal values and this step size increases with increasing input level. As a result the signal-to-quantization noise ratio improves for smaller signal values at the cost of that for larger signal levels (see reference [6]). After applying sampling and quantization to a continuous stochastic process we actually have a discrete random sequence, but, as mentioned in Chapter 1, these processes are simply special cases of the discrete-time processes. 3.6 SPECTRUM OF DISCRETE-TIME PROCESSES As usual, the calculation of the power spectrum has to start by considering the autocorrelation function. For the wide-sense stationary discrete-time process X½nŠ we can write RXX½mŠ ¼ E½X½nŠ X½n þ mŠŠ ð3:62Þ The relation to the continuous stochastic process is mX ¼1 RXX½mŠ ¼ RXXðmTsÞ ðt À mTsÞ m¼À1 ð3:63Þ From this equation the power spectral density of X½mŠ follows by Fourier transformation: mX ¼1 SXXð!Þ ¼ RXX½mŠ expðÀj!mTsÞ m¼À1 ð3:64Þ Thus, the spectrum is a periodic function (see Figure 3.8) with period 2=Ts. Such a periodic function can be described by means of its Fourier series coefficients [7,10] RXX ½mŠ ¼ Ts 2p Z p=Ts Àp=Ts SXX ð!Þ expðj!mTsÞ d! ð3:65Þ In particular, we find for the power of the process PX ¼ E½X2½nŠŠ ¼ RXX ½0Š ¼ Z Ts p=Ts 2p Àp=Ts SXXð!Þ d! ð3:66Þ For ease of calculation it is convenient to introduce the z-transform of RXX½mŠ, which is defined as ~SXXðzÞ ¼4 mX ¼1 RXX½mŠ zÀm m¼À1 ð3:67Þ 58 SPECTRA OF STOCHASTIC PROCESSES The z-transform is more extensively dealt with in Subsection 4.6.2. Comparing this latter expression to Equation (3.64) reveals that the delay operator z equals expðj!TsÞ and consequently ~SXXðexpðj!mTsÞÞ ¼ SXXð!Þ ð3:68Þ Example 3.7: Let us consider a wide-sense stationary discrete-time process X½nŠ with the autocorrelation sequence RXX½mŠ ¼ ajmj; with jaj < 1 ð3:69Þ The spectrum expressed in z-transform notation is mX ¼1 mX ¼À1 mX ¼1 ~SXXðzÞ ¼ ajmjzÀm ¼ aÀmzÀm þ amzÀm m¼À1 m¼À1 m¼0 ¼ 1 az À az þ z z À a ¼ ð1=a 1=a À a þ aÞ À ðz þ 1=zÞ ð3:70Þ From this expression the spectrum in the frequency domain is easily derived by replacing z with expðj!TsÞ: SXX ð!Þ ¼ ð1=a þ 1=a À aÞ À 2 a cosð!TsÞ ð3:71Þ & It can be seen that the procedure given here to develop the autocorrelation sequence and spectrum of a discrete-time process can equally be applied to derive cross-correlation sequences and corresponding cross-power spectra. We leave further elaboration on this subject to the reader. 3.7 SUMMARY The power spectral density function, or power spectrum, of a stochastic process is defined as the Fourier transform of the autocorrelation function. This spectrum shows how the total power of the process is distributed over the various frequencies. Definitions have been given of the bandwidth of stochastic processes. It appears that on modulation the power spectrum is split up into two parts of identical shape as the original unmodulated spectrum: one part is concentrated around the modulation frequency and the other part around minus the modulation frequency. This implies that we use a description based on double-sided spectra. This is very convenient from a mathematical point of view. From a physical point PROBLEMS 59 of view negative frequency values have no meaning. When changing to the physical interpretation, the contributions of the power spectrum at negative frequencies are mirrored with respect to the y axis and the values are added to the values at the corresponding positive frequencies. The sampling theorem is redefined for stochastic processes. Therefore continuous stochastic processes can be converted into discrete-time processes without information becoming lost. For processing signals using a digital signal processor (DSP), still another step is needed, namely analogue-to-digital conversion. This introduces errors that cannot be restored in the digital-to-analogue reconstruction. These errors are calculated and expressed in terms of the signal-to-noise ratio. Finally, the autocorrelation sequence and power spectrum of discrete-time processes are derived. 3.8 PROBLEMS 3.1 A wide-sense stationary process XðtÞ has the autocorrelation function  RXXðÞ ¼ A exp À jj T (a) Calculate the power spectrum SXXð!Þ. (b) Calculate the power of XðtÞ using the power spectrum. (c) Check the answer to (b) using Equation (3.8), i.e. based on the autocorrelation function evaluated at  ¼ 0. (d) Use Matlab to plot the spectrum for A ¼ 3 and T ¼ 4. 3.2 Consider the process XðtÞ, of which the autocorrelation function is given in Problem 2.14. (a) Calculate the power spectrum SXXð!Þ. Make a plot of it using Matlab. (b) Calculate the power of XðtÞ using the power spectrum. (c) Calculate the mean value of XðtÞ from the power spectrum. (d) Calculate the variance of XðtÞ using the power spectrum. (e) Check the answers to (b), (c) and (d) using the answers to Problem 2.14. (f) Calculate the lowest frequency where the spectrum becomes zero. By means of Matlab calculate the relative amount of a.c. power in the frequency band between zero and this first null. 3.3 A wide-sense stationary process has a power spectral density function  2; 0 j!j < 10=ð2pÞ SXXð!Þ ¼ 0; elsewhere (a) Calculate the autocorrelation function RXXðÞ. (b) Use Matlab to plot the autocorrelation function. (c) Calculate the power of the process, both via the power spectrum and the autocorrelation function. 60 SPECTRA OF STOCHASTIC PROCESSES 3.4 Consider the process YðtÞ ¼ A2 sin2ð!0t À ÂÞ, with A and !0 constants and  a random variable that is uniformly distributed on ð0; 2pŠ. In Problem 2.8 we calculated its autocorrelation function. (a) Calculate the power spectrum SYY ð!Þ. (b) Calculate the power of YðtÞ using the power spectrum. (c) Check the answer to (b) using Equation (3.8). 3.5 Reconsider Problem 2.9 and insert !0 ¼ 2p. (a) Calculate the magnitude of a few spectral lines of the power spectrum by means of Matlab. (b) Based on (a), calculate an approximate value of the total power and check this on the basis of the autocorrelation function. (c) Calculate and check the d.c. power. 3.6 Answer the same questions as in Problem 3.5, but now for the process given in Problem 2.11 when inserting T ¼ 1. 3.7 A and B are random variables. These variables are used to create the process XðtÞ ¼ A cos !0t þ B sin !0t, with !0 a constant. (a) Assume that A and B are uncorrelated, have zero means and equal variances. Show that in this case XðtÞ is wide-sense stationary. (b) Derive the autocorrelation function of XðtÞ. (c) Derive the power spectrum of XðtÞ. Make a sketch of it. 3.8 A stochastic process is given by XðtÞ ¼ A cosðt À ÂÞ, where A is a real constant,  a random variable with probability density function fð!Þ and  a random variable that is uniformly distributed on the interval (0,2p], independent of . Show that the power spectrum of XðtÞ is SXX ð!Þ ¼ pA2 2 ½ fð!Þ þ fðÀ!ފ 3.9 A and B are real constants and XðtÞ is a wide-sense stationary process. Derive the power spectrum of the process YðtÞ ¼ A þ B XðtÞ. 3.10 Can each of the following functions be the autocorrelation function of a wide-sense stationary process XðtÞ? (a) RXXð Þ ¼ ðÞ. (b) RXXðÞ ¼ rectðÞ. (c) RXXð Þ ¼ triðÞ. For definitions of the functions ðÁÞ, rectðÁÞ and triðÁÞ see Appendix E. 3.11 Consider the process given in Problem 3.1. Based on process XðtÞ of that problem another process YðtÞ is produced, such that  SYY ð!Þ ¼ SXXð!Þ; j!j < cð1=TÞ 0; elsewhere where c is a constant. PROBLEMS 61 (a) Calculate the r.m.s. bandwidth of YðtÞ. (b) Consider the consequences, both for SYYð!Þ and the r.m.s. bandwidth, when c ! 1. 3.12 For the process XðtÞ it is found that RXXðÞ ¼ A exp½À2=ð22ފ, with A and  positive constants. (a) Derive the expression for the power spectrum of XðtÞ. (b) Calculate the r.m.s. bandwidth of XðtÞ. 3.13 Derive the cross-power spectrum of the processes given in Problem 2.9. 3.14 A stochastic process is defined by WðtÞ ¼ AXðtÞ þ BYðtÞ, with A and B real constants and XðtÞ and YðtÞ jointly wide-sense stationary processes. (a) Calculate the power spectrum of WðtÞ. (b) Calculate the cross-power spectra SXW ð!Þ and SYW ð!Þ. (c) Derive SWW ð!Þ when XðtÞ and YðtÞ are orthogonal. (d) Derive SWW ð!Þ when XðtÞ and YðtÞ are independent. (e) Derive SWW ð!Þ when XðtÞ and YðtÞ are independent and have mean values of zero. (f) Derive SWW ð!Þ when XðtÞ and YðtÞ are uncorrelated. 3.15 A wide-sense stationary noise process NðtÞ has a power spectrum as given in Figure 3.11. This process is added to an harmonic random signal SðtÞ ¼ 3 cosð8t À ÈÞ and the sum SðtÞ þ NðtÞ is applied to one of the inputs of a product modulator. To the second input of this modulator another harmonic process XðtÞ ¼ 2 cosð8t À ÂÞ is applied. The random variables È and  are independent, but have the same uniform distribution on the interval ½0; 2pÞ. Moreover, these random variables are independent of the process NðtÞ. The output of the product modulator is connected to an ideal lowpass filter with a cut-off angular frequency !c ¼ 5. (a) Make a sketch of the spectrum of the output of the modulator. (b) Sketch the spectrum at the output of the lowpass filter. (c) Calculate the d.c. power at the output of the filter. (d) The output signal is defined as that portion of the output due to SðtÞ. The output noise is defined as that portion of the output due to NðtÞ. Calculate the output signal power and the output noise power, and the ratio between the two (called the signal-to-noise ratio). SNN (ω) 1 –10 –5 0 5 Figure 3.11 10 ω 62 SPECTRA OF STOCHASTIC PROCESSES 3.16 The two independent processes XðtÞ and YðtÞ are applied to a product modulator. Process XðtÞ is a wide-sense stationary process with power spectral density  1; SXXð!Þ ¼ 0; j!j WX j!j > WX The process YðtÞ is an harmonic carrier, but both its phase and frequency are independent random variables YðtÞ ¼ cosðt À ÂÞ where the phase  is uniformly distributed on the interval ½0; 2pÞ and the carrier frequency is uniformly distributed on the interval ð!0 À WY =2 <  < !0 þ WY =2Þ and with !0 > WY =2 ¼ constant. (a) Calculate the autocorrelation function of the process YðtÞ. (b) Is YðtÞ wide-sense stationary? (c) If so, determine and sketch the power spectral density SYY ð!Þ. (d) Determine and sketch the power spectral density of the product ZðtÞ ¼ XðtÞYðtÞ. Assume that WY =2 þ WX < !0 and WX < WY =2. 3.17 The following sequence of numbers represents sample values of a band-limited signal: . . . ; 0; 5; 10; 20; 40; 0; À20; À15; À10; À5; 0; . . . All other samples are zero. Use Matlab to reconstruct and graphically represent the signal. 3.18 In the case of ideal sampling, the sampled version of the signal f ðtÞ is represented by nX ¼1 fsðtÞ ¼ f ðnTsÞ ðt À nTsÞ n¼À1 In so-called ‘flat-top sampling’ the samples are presented by the magnitude of rectangular pulses, i.e. fsðtÞ ¼ nX ¼1 n¼À1 f ðnTsÞ  t rect  À nTs s where s < Ts. (See Appendix E for the definition of the rectðÁÞ function.) Investigate the effect of using these rectangular pulses on the Fourier transform of the recovered signal. 3.19 A voice channel has a spectrum that runs up to 3.4 kHz. On sampling, a guard band (i.e. the distance between adjacent spectral replicas after sampling) of 1.2 kHz has to be taken in account. PROBLEMS 63 (a) What is the minimum sampling rate? (b) When the samples are coded by means of a linear sampler of 8 bits, calculate the bit rate of a digitized voice channel. (c) What is the maximum signal-to-noise ratio that can be achieved for such a voice channel? (d) With how many dB will the signal-to-noise reduce when only half of the dynamic range is used by the signal? 3.20 A stochastic process XðtÞ has the power spectrum SXX ¼ 1 1 þ !2 This process is sampled, and since it is not band-limited the adjacent spectral replicas will overlap. If the spill-over power, i.e. the amount of power that is in overlapping frequency ranges, between adjacent replicas has to be less than 10% of the total power, what is the minimum sampling frequency? 3.21 The discrete-time process X½nŠ is wide-sense stationary and RXX½1Š ¼ RXX½0Š. Show that RXX½mŠ ¼ RXX½0Š for all m. 3.22 The autocorrelation sequence of a discrete-time wide-sense stationary process X½nŠ is  RXX½mŠ ¼ 1 À 0:2jmj; 0; jmj 4 jmj > 4 Calculate the spectrum SXXð!Þ and make a plot of it using Matlab. 4 Linear Filtering of Stochastic Processes In this chapter we will investigate what the influence will be on the main parameters of a stochastic process when filtered by a linear, time-invariant filter. In doing so we will from time to time change from the time domain to the frequency domain and vice versa. This may even happen during the course of a calculation. From Fourier transform theory we know that both descriptions are dual and of equal value, and basically there is no difference, but a certain calculation may appear to be more tractable or simpler in one domain, and less tractable in the other. In this chapter we will always assume that the input to the linear, time-invariant filter is a wide-sense stationary process, and the properties of these processes will be invoked several times. It should be stressed that the presented calculations and results may only be applied in the situation of wide-sense stationary input processes. Systems that are non-linear or timevariant are not considered, and the same holds for input processes that do not fulfil the requirements for wide-sense stationarity. We start by summarizing the fundamentals of linear time-invariant filtering. 4.1 BASICS OF LINEAR TIME-INVARIANT FILTERING In this section we will summarize the theory of continuous linear time-invariant filtering. For the sake of simplicity we consider only single-input single-output (SISO) systems. For a more profound treatment of this theory see references [7] and [10]. The generalization to multiple-input multiple-output (MIMO) systems is straightforward and requires a matrix description. Let us consider a general system that converts a certain input signal xðtÞ into the corresponding output signal yðtÞ. We denote this by means of the general hypothetical operator T½ÁŠ as follows (see also Figure 4.1(a)): yðtÞ ¼ T½xðtފ ð4:1Þ Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 66 LINEAR FILTERING OF STOCHASTIC PROCESSES x (t ) T [.] y (t ) (a) x (t )=exp(jωt ) h (t ) H (ω) y (t)=A exp{ j(ωt +φ)} (b) Figure 4.1 (a) General single-input single-output (SISO) system; (b) linear time-invariant (LTI) system Next we limit our treatment to linear systems and denote this by means of L½ÁŠ as follows: yðtÞ ¼ L½xðtފ ð4:2Þ The definition of linearity of a system is as follows. Suppose a set of input signals fxnðtÞg causes a corresponding set of output signals fynðtÞg. Then a system is said to be linear if any arbitrary linear combination of inputs causes the same linear combination of corresponding outputs, i.e. if : X xnðtÞ )X ynðtÞ then : anxnðtÞ ) anynðtÞ n n ð4:3Þ with an arbitrary constants. In the notation of Equation (4.2), X !X X yðtÞ ¼ L anxnðtÞ ¼ anL½xnðtފ ¼ anynðtÞ n n n ð4:4Þ A system is said to be time-invariant if a shift in time of the input causes a corresponding shift in the output. Therefore, if : xnðtÞ ) ynðtÞ then : xnðt À Þ ) ynðt À Þ ð4:5Þ for arbitrary . Finally, a system is linear time-invariant (LTI) if it satisfies both conditions given by Equations (4.3) and (4.5), i.e. if : X xnðtÞ ) ynðtÞ X then : xðtÞ ¼ anxnðt À nÞ ) yðtÞ ¼ anynðt À nÞ n n ð4:6Þ BASICS OF LINEAR TIME-INVARIANT FILTERING 67 It can be proved [7] that complex exponential time functions, i.e. sine and cosine waves, are so-called eigenfunctions of linear time-invariant systems. An eigenfunction can physically be interpreted as a function that preserves its shape on transmission, i.e. a sine/cosine remains a sine/cosine, but its amplitude and/or phase may change. When these changes are known for all frequencies then the system is completely specified. This specification is done by means of the complex transfer function of the linear time-invariant system. If the system is excited with a complex exponential xðtÞ ¼ expðj!tÞ ð4:7Þ and the corresponding output is yðtÞ ¼ A exp½ jð!t þ ’ފ ð4:8Þ with A a real constant, then the transfer function equals Hð!Þ ¼ yxððttÞÞxðtÞ¼expðj!tÞ ð4:9Þ From Equations (4.7) to (4.9) the amplitude and the phase angle of the transfer function follow: jHð!Þj ¼ A ff Hð!Þ ¼ ’ ð4:10Þ When in Equation (4.9), ! is given all values from À1 to 1, the transfer is completely known. As indicated in that equation the amplitude of the input xðtÞ is taken as unity and the phase zero for all frequencies. All observed complex values of yðtÞ are then presented by the function Yð!Þ. Since the input Xð!Þ was taken as unity for all frequencies, Equation (4.9) is in that case written as Hð!Þ ¼ Yð!Þ ! Yð!Þ ¼ Hð!Þ Â 1 1 ð4:11Þ From Fourier theory we know that the multiplication in the right-hand side of the latter equation is written in the time domain as a convolution [7,10]. Moreover, the inverse transform of 1 is a  function. Since a  function is also called an impulse, the time domain LTI system response following from Equation (4.11) is called the impulse response. We may therefore conclude that the system impulse response hðtÞ and the system transfer function Hð!Þ constitute a Fourier transform pair. When the transfer function is known, the response of an LTI system to an input signal can be calculated. Provided that the input signal xðtÞ satisfies the Dirichlet conditions [10], its Fourier transform Xð!Þ exists. However, this frequency domain description of the signal is equivalent to decomposing the signal into complex exponentials, which in turn are eigenfunctions of the LTI system. This allows multiplication of Xð!Þ by Hð!Þ to find Yð!Þ, being the Fourier transform of output yðtÞ; namely by taking the inverse transform of Yð!Þ, the signal yðtÞ is reconstructed from its complex exponential components. This 68 LINEAR FILTERING OF STOCHASTIC PROCESSES justifies the use of Fourier transform theory to be applied to the transmission of signals through LTI signals. This leads us to the following theorem. Theorem 6 If a linear time-invariant system with an impulse response hðtÞ is excited by an input signal xðtÞ, then the output is Z1 yðtÞ ¼ hðÞ xðt À Þ d ð4:12Þ À1 with the equivalent frequency domain description Yð!Þ ¼ Hð!Þ Xð!Þ ð4:13Þ where Xð!Þ and Yð!Þ are the Fourier transforms of xðtÞ and yðtÞ, respectively, and Hð!Þ is the Fourier transform of the impulse response hðtÞ. The two presentations of Equations (4.12) and (4.13) are so-called dual descriptions; i.e. both are complete and either of them is fully determined by the other one. If an important condition for physical realizability of the LTI system is taken into account, namely causality, then the impulse response will be zero for t < 0 and the lower bound of the integral in Equation (4.12) changes into zero. This theorem is the main result we need to describe the filtering of stochastic processes by an LTI system, as is done in the sequel. 4.2 TIME DOMAIN DESCRIPTION OF FILTERING OF STOCHASTIC PROCESSES Let us now consider the transmission of a stochastic process through an LTI system. Obviously, we may formally apply the time domain description given by Equation (4.12) to calculate the system response of a single realization of the ensemble. However, a frequency domain description is not always possible. Apart from the fact that realizations are often not explicitly known, it may happen that they do not satisfy the Dirichlet conditions. Therefore, we start by characterizing the filtering in the time domain. Later on the frequency domain description will follow from this. 4.2.1 The Mean Value of the Filter Output The impulse response of the linear, time-invariant filter is denoted by hðtÞ. Let us consider the ensemble of input realizations and call this input process XðtÞ and the corresponding output process YðtÞ. Then the relation between input and output is formally described by the convolution Z1 YðtÞ ¼ hðÞXðt À Þ d ð4:14Þ À1 TIME DOMAIN DESCRIPTION OF FILTERING OF STOCHASTIC PROCESSES 69 When the input process XðtÞ is wide-sense stationary, then the mean value of the output signal is written as Z1 ! Z1 E½Yðtފ ¼ E hðÞXðt À Þ d ¼ hðÞE½Xðt À ފ d À1 Z1 À1 ¼ YðtÞ ¼ XðtÞ hðÞ d ¼ XðtÞ Hð0Þ À1 ð4:15Þ where Hð!Þ is the Fourier transform of hðtÞ. From Equation (4.15) it follows that the mean value of YðtÞ equals the mean value of XðtÞ multiplied by the value of the transfer function for the d.c. component. This value is equal to the area under the curve of the impulse response function hðtÞ. This conclusion is based on the property of XðtÞ at least being stationary of the first order. 4.2.2 The Autocorrelation Function of the Output The autocorrelation function of YðtÞ is found using the definition of Equation (2.13) and Equation (4.14): RYY ðt; t þ Þ ¼ E½YðtÞ Yðt Z1 þ  ފ Z1 ! ¼E hð1ÞXðt À 1Þ d1 hð2ÞXðt þ  À 2Þ d2 À1 À1 Z1Z ¼ E½Xðt À 1Þ Xðt þ  À 2ފhð1Þhð2Þ d1 d2 À1 ð4:16Þ Invoking XðtÞ as wide-sense stationary reduces this expression to Z1Z RYY ðÞ ¼ RXXð þ 1 À 2ފhð1Þhð2Þ d1 d2 À1 ð4:17Þ and the mean squared value of YðtÞ reads Z1Z E½Y2ðtފ ¼ Y2ðtÞ ¼ RYY ð0Þ ¼ RXXð1 À 2Þhð1Þhð2Þ d1 d2 À1 ð4:18Þ From Equations (4.15) and (4.17) it is concluded that YðtÞ is wide-sense stationary when XðtÞ is wide-sense stationary, since neither the right-hand member of Equation (4.15) nor that of Equation (4.17) depends on t. Equation (4.17) may also be written as RYY ðÞ ¼ RXXðÞ Ã hðÞ Ã hðÀ Þ where the symbol à represents the convolution operation. ð4:19Þ 70 LINEAR FILTERING OF STOCHASTIC PROCESSES 4.2.3 Cross-Correlation of the Input and Output The cross-correlation of XðtÞ and YðtÞ is found using Equations (2.46) and (4.14): Z1 ! RXY ðt; t þ Þ ¼4 E½XðtÞ Yðt þ ފ ¼ E XðtÞ hðÞXðt þ  À Þ d Z1 À1 ¼ E½XðtÞ Xðt þ  À ފ hðÞ d À1 ð4:20Þ In the case where XðtÞ is wide-sense stationary Equation (4.20) reduces to Z1 RXY ðÞ ¼ RXXð À ÞhðÞ d À1 ð4:21Þ This expression may also be presented as the convolution of RXXðÞ and hðÞ: RXY ð Þ ¼ RXXð Þ Ã hð Þ ð4:22Þ In a similar way the following expression can be derived: Z1 RYXðÞ ¼ RXXð þ ÞhðÞ d ¼ RXXðÞ Ã hðÀÞ À1 ð4:23Þ From Equations (4.21) and (4.23) it is concluded that the cross-correlation functions do not depend on the absolute time parameter t. Earlier we concluded that YðtÞ is wide-sense stationary if XðtÞ is wide-sense stationary. Now we conclude that XðtÞ and YðtÞ are jointly wide-sense stationary if the input process XðtÞ is wide-sense stationary. Substituting Equation (4.21) into Equation (4.17) reveals the relation between the autocorrelation function of the output and the cross-correlation between the input and output: Z1 RYY ðÞ ¼ RXY ð þ 1Þhð1Þ d1 À1 ð4:24Þ or, presented differently, RYY ðÞ ¼ RXY ð Þ Ã hðÀ Þ ð4:25Þ In a similar way it follows by substitution of Equation (4.23) into Equation (4.17) that Z1 RYY ð Þ ¼ RYXð À 2Þhð2Þ d2 ¼ RYXð Þ Ã hð Þ À1 ð4:26Þ Example 4.1: An important application of the cross-correlation function as given by Equation (4.22) consists of the identification of a linear system. If for the input process a white noise process, i.e. a process with a constant value of the power spectral density, let us say of magnitude SPECTRA OF THE FILTER OUTPUT 71 N0=2, is selected then the autocorrelation function of that process becomes N0 ðÞ=2. This makes the convolution very simple, since the convolution of a  function with another function results in this second function itself. Thus, in that case the cross-correlation function of the input and output yields RXY ðÞ ¼ N0 hðÞ=2. Apart from a constant N0=2, the cross-correlation function equals the impulse response of the linear system; in this way we have found a method to measure this impulse response. & 4.3 SPECTRA OF THE FILTER OUTPUT In the preceding sections we described the output process of a linear time-invariant filter in terms of the properties of the input process. In doing so we used the time domain description. We concluded that in case of a wide-sense stationary input process the corresponding output process is wide-sense stationary as well, and that the two processes are jointly wide-sense stationary. This offers the opportunity to apply the Fourier transform to the different correlation functions in order to arrive at the spectral description of the output process and the relationship between the input and output processes. It must also be stressed that in this section only wide-sense stationary input processes will be considered. The first property we are interested in is the spectral density of the output process. Using what has been derived in Section 4.2.2, this is easily revealed by transforming Equation (4.19) to the frequency domain. If we remember that the impulse response hðÞ is a real function and thus the Fourier transform of hðÀÞ equals HÃð!Þ, then the next important statement can be exposed. Theorem 7 If a wide-sense stationary process XðtÞ, with spectral density SXXð!Þ, is applied to the input of a linear, time-invariant filter with the transfer function Hð!Þ, then the corresponding output process YðtÞ is a wide-sense stationary process as well, and the spectral density of the output reads SYY ð!Þ ¼ SXXð!Þ Hð!Þ HÃð!Þ ¼ SXXð!Þ jHð!Þj2 ð4:27Þ The mean power of the output process is written as PY ¼ RYY ð0Þ ¼ 1Z1 2p À1 SXXð!Þ jHð!Þj2 d! ð4:28Þ Example 4.2: Consider the RC network given in Figure 4.2. Then the voltage transfer function of this network is written as Hð!Þ ¼ 1 þ 1 j!RC ð4:29Þ 72 LINEAR FILTERING OF STOCHASTIC PROCESSES R C Figure 4.2 RC network If we assume that the network is excited by a white noise process with spectral density of N0=2 and taking the modulus squared of Equation (4.29), then the output spectral density reads SYY ð!Þ ¼ 1 N0=2 þ ð!RCÞ2 ð4:30Þ For the power in the output process it is found that PY ¼ 1Z1 2p À1 1 N0=2 þ ð!RCÞ2 d! ¼ N0 4pRC arctanð!RCÞ1 À1 ¼ N0 4RC ð4:31Þ & In an earlier stage we found the power of a wide-sense stationary process in an alternative way, namely the value of the autocorrelation function at  ¼ 0 (see, for instance, Equation (4.16)). The power can also be calculated using that procedure. However, in order to be able to calculate the autocorrelation function of the output YðtÞ we need the probability density function of XðtÞ in order to evaluate a double convolution or we need the probability density function of YðtÞ. Finding this latter function we meet two main obstacles: firstly, measuring the probability density function is much more difficult than measuring the power density function and, secondly, the probability density function of YðtÞ by no means follows in a simple way from that of XðtÞ. This latter statement has one important exception, namely if the probability density function of XðtÞ is Gaussian then the probability density function of YðtÞ is Gaussian as well (see Section 2.3). However, calculating the mean and variance of YðtÞ, which are sufficient to determine the Gaussian density, using Equations (4.15) and (4.28) is a simpler and more convenient method. From Equations (4.22) and (4.23) the cross-power spectra are deduced: SXY ð!Þ ¼ SXXð!Þ Hð!Þ SYXð!Þ ¼ SXXð!Þ HðÀ!Þ ¼ SXXð!ÞHÃð!Þ ð4:32Þ ð4:33Þ We are now in a position to give the proof of Equation (3.5). Suppose that SXXð!Þ has a negative value for some arbitrary ! ¼ !0. Then a small interval ð!1; !2Þ about !0 is found, such that (see Figure 4.3(a)) SXXð!Þ < 0; for !1 < j!j < !2 ð4:34Þ SXX (ω) H (ω) 1 SYY (ω) SPECTRA OF THE FILTER OUTPUT 73 ω1 ω2 (a) ω (b) (c) Figure 4.3 Noise filtering Now consider an ideal bandpass filter with the transfer function (see Figure 4.3(b)) & Hð!Þ ¼ 1; 0; !1 < j!j < !2 for all remaining values of ! ð4:35Þ If the process XðtÞ, with the power spectrum given in Figure 4.3(a), is applied to the input of this filter, then the spectrum of the output YðtÞ is as presented in Figure 4.3(c) and is described by & SYY ð!Þ ¼ SXXð!Þ; !1 < j!j < !2 0; for all remaining values of ! ð4:36Þ so that SYY ð!Þ 0; for all ! However, this is impossible as (see Equations (4.28) and (3.8)) PY ¼ RYY ð0Þ ¼ 1Z1 2p À1 SYY ð!Þ d! ! 0 ð4:37Þ ð4:38Þ 74 LINEAR FILTERING OF STOCHASTIC PROCESSES This contradiction leads to the conclusion that the starting assumption SXXð!Þ < 0 must be wrong. 4.4 NOISE BANDWIDTH In this section we present a few definitions and concepts related to the bandwidth of a process or a linear, time-invariant system (filter). 4.4.1 Band-Limited Processes and Systems A process XðtÞ is called a band-limited process if SXXð!Þ ¼ 0 outside certain regions of the ! axis. For a band-limited filter the same definition can be used, provided that SXXð!Þ is replaced by Hð!Þ. A few special cases of band-limited processes and systems are considered in the sequel. 1. A process is called a lowpass process or baseband process if & Sð!Þ 6¼ 0; ¼ 0; j!j < W j!j > W ð4:39Þ 2. A process is called a bandpass process if (see Figure 4.4) & Sð!Þ 6¼ 0; ¼ 0; !0 À W1 j!j !0 À W1 þ W for all remaining values of ! ð4:40Þ with 0 < W1 < !0 ð4:41Þ 3. A system is called a narrowband system if the bandwidth of that system is small compared to the frequency range over which the spectrum of the input process extends. A S(ω) 0 ω0 − W1 ω0 ω0 − W1 + W ω W Figure 4.4 The spectrum of a bandpass process (only the region ! ! 0 is shown) NOISE BANDWIDTH 75 narrowband bandpass process is a process for which the bandwidth is much smaller than its central frequency, i.e. (see Figure 4.4) W ( !0 ð4:42Þ The following points should be noted:  The definitions of processes 1 and 2 can also be used for systems if Sð!Þ is replaced by Hð!Þ.  In practical systems or processes the requirement that the spectrum or transfer function is zero in a certain region cannot exactly be met in a strict mathematical sense. Nevertheless, we will maintain the given names and concepts for those systems and processes for which the transfer function or spectrum has a negligibly low value in a certain frequency range.  The spectrum of a bandpass process is not necessarily symmetrical about !0. 4.4.2 Equivalent Noise Bandwidth Equation (4.28) is often used for practical applications. For that reason there is a need for a simplified calculation method in order to compute the noise power at the output of a filter. In this section we will introduce such a simplification. To that end consider a lowpass system with the transfer function Hð!Þ. Assume that the spectrum of the input process equals N0=2 for all !, with N0 a positive, real constant (such a spectrum is called a white noise spectrum). The power at the output of the filter is calculated using Equation (4.28): PY ¼ 1Z1 2p À1 N0 2 jHð!Þj2 d! ð4:43Þ Now define an ideal lowpass filter as & HIð!Þ ¼ Hð0Þ; 0; j!j WN j!j > WN ð4:44Þ where WN is a positive constant chosen such that the noise power at the output of the ideal filter is equal to the noise power at the output of the original (practical) filter. WN therefore follows from the equation 1 2p Z1 À1 N0 2 jHð!Þj2 d! ¼ 1 2p Z WN ÀWN N0 2 jHð0Þj2 d! ð4:45Þ If we consider jHð!Þj2 to be an even function of !, then solving Equation (4.45) for WN yields WN ¼ R1 0 jH ð!Þj2d! jHð0Þj2 ð4:46Þ 76 LINEAR FILTERING OF STOCHASTIC PROCESSES |H I(ω)|2 |H (ω)|2 –WN 0 WN ω Figure 4.5 Equivalent noise bandwidth of a filter characteristic WN is called the equivalent noise bandwidth of the filter with the transfer function Hð!Þ. In Figure 4.5 it is indicated graphically how the equivalent noise bandwidth is determined. The solid curve represents the practical characteristic and the dashed line the ideal rectangular one. The equivalent noise bandwidth is such that in this picture the dashed area equals the shaded area. From Equations (4.43) and (4.45) it follows that the output power of the filter can be written as PY ¼ N0 2p jHð0Þj2 WN ð4:47Þ Thus, it can be shown for the special case of white input noise that the integral of Equation (4.43) is reduced to a product and the filter can be characterized by means of a single number WN as far as the noise filtering behaviour is concerned. Example 4.3: As an example of the equivalent noise bandwidth let us again consider the RC network presented in Figure 4.2. Using the definition of Equation (4.46) and the result of Example 4.2 (Equation (4.29)) it is found that WN ¼ p=ð2RCÞ. This differs from the 3 dB bandwidth by a factor of p=2. It may not come as a surprise that the equivalent noise bandwidth of a circuit differs from the 3 dB bandwidth, since the definitions differ. On the other hand, both bandwidths are proportional to 1=ðRCÞ. & Equation (4.47) can often be used for the output of a narrowband lowpass filter. For such a system, which is analogous to Equation (4.43), the output power reads PY % 1Z1 2p À1 SXXð0Þ jHð!Þj2 d! ð4:48Þ and thus PY % SXX ð0Þ p jH ð0Þj2WN ð4:49Þ SPECTRUM OF A RANDOM DATA SIGNAL 77 The above calculation may also be applied to bandpass filters. Then it follows that WN ¼ R1 0 jHð!Þj2 d! jHð!0Þj2 ð4:50Þ Here !0 is a suitably chosen but arbitrary frequency in the passband of the bandpass filter, for instance the centre frequency or the frequency where jHð!Þj attains its maximum value. The noise power at the output is written as PY ¼ N0 2p jH ð!0 Þj2WN ð4:51Þ When once again the input spectrum is approximately constant within the passband of the filter, which often happens in narrowband bandpass filters, then PY % SXX ð!0 Þ p jH ð!0 Þj2 WN ð4:52Þ In this way we end up with rather simple expressions for the noise output of linear timeinvariant filters. 4.5 SPECTRUM OF A RANDOM DATA SIGNAL This subject is dealt with here since for the derivation we need results from the filtering of stochastic processes, as dealt with in the preceding sections of this chapter. Let us consider the random data signal X XðtÞ ¼ A½nŠ pðt À nTÞ n ð4:53Þ where the data sequence is produced by making a random selection out of the possible values of A½nŠ for each moment of time nT. In the binary case, for example, we may have A½nŠfÀ1; þ1g. The sequence A½nŠ is supposed to be wide-sense stationary, where A½nŠ and A½kŠ in general will be correlated according to  à  Ã à E A½nŠ A½kŠ ¼ E ½A½nŠ A n þ mŠ ¼ E A½nŠ A½n À mŠ ¼ R½mŠ ð4:54Þ The data symbols amplitude modulate the waveform pðtÞ. This waveform may extend beyond the boundaries of a bit interval. The random data signal XðtÞ constitutes a cyclo-stationary process. We define a random variable  which is uniformly distributed on the interval ð0; TŠ and which is supposed to be independent of the data A½nŠ; this latter assumption sounds reasonable. Using this random variable and the process XðtÞ we define the new process X XðtÞ ¼4 Xðt À ÂÞ ¼ A½nŠ pðt À nT À ÂÞ n ð4:55Þ 78 LINEAR FILTERING OF STOCHASTIC PROCESSES Invoking Theorem 1 (see Section 2.2.2) we may conclude that this latter process is stationary. We model the process XðtÞ as resulting from exciting a linear time-invariant system having the impulse response hðtÞ ¼ pðtÞ by the input process X YðtÞ ¼ A½nŠ ðt À nT À ÂÞ ð4:56Þ n The autocorrelation function of YðtÞ is RYY ðÞ ¼ E½YðtÞ Yðt þ ފ XX ! ¼E A½nŠA½kŠ ðt À nT À ÂÞ ðt À kT þ  À ÂÞ X X n k à ¼ E A½nŠ A½kŠ E½ðt À nT À ÂÞ ðt À kT þ  À Âފ ¼ nk XX R½k À nŠ Z 1 T ðt À nT À Þ ðt þ  À kT À Þ d nk T0 ð4:57Þ For all values of T; t;  and n there will be only one single value of k for which both ðt À nT À Þ and ðt þ  À kT À Þ will be found in the interval 0 <  T. This means that actually the integral in Equation (4.57) is the convolution of two  functions,which is well defined (see reference [7]). Applying the basic definition of  functions (see Appendix E) yields RYY ð Þ ¼ X m R½mŠ T ð þ mT Þ ð4:58Þ where m ¼ k À n. The autocorrelation function of YðtÞ is presented in Figure 4.6. Finally, the autocorrelation function of XðtÞ follows: RXXðÞ ¼ RYY ð Þ Ã hðÞ Ã hðÀÞ ¼ RYY ðÞ Ã pð Þ Ã pðÀ Þ ð4:59Þ The spectrum of the random data signal is found by Fourier transforming Equation (4.59): SXXð!Þ ¼ SYY ð!Þ jPð!Þj2 ð4:60Þ RYY (τ) ... ... –2T –T 0 T 2T τ Figure 4.6 The autocorrelation function of the process YðtÞ consisting of a sequence of  functions SPECTRUM OF A RANDOM DATA SIGNAL 79 where Pð!Þ is the Fourier transform of pðtÞ. Using this latter equation and Equation (4.58) the following theorem can be stated. Theorem 8 The spectrum of a random data signal reads SXX ð!Þ ¼ jPð!Þj2 T X 1 m(¼À1 R½mŠ expðj!mT Þ ) ¼ jPð!Þj2 X 1 R½0Š þ 2 R½mŠ cosð!mTÞ T m¼1 ð4:61Þ where R½mŠ are the autocorrelation values of the data sequence, Pð!Þ is the Fourier transform of the data pulses and T is the bit time. This result, which was found by applying filtering to a stochastic process, is of great importance when calculating the spectrum of digital baseband or digitally modulated signals in communications, as will become clear from the examples in the sequel. When applying this theorem, two cases should clearly be distinguished, since they behave differently, both theoretically and as far as the practical consequences are concerned. We will deal with the two cases by means of examples. Example 4.4: The first case we consider is the situation where the mean value of XðtÞ is zero and consequently the autocorrelation function of the sequence A½nŠ has in practice a finite extent. Let us suppose that the summation in Equation (4.61) runs in that case from 1 to M. As an important practical example of this case we consider the so-called polar NRZ signal, where NRZ is the abbreviation for non-return to zero, which reflects the behaviour of the data pulses. Possible values of A½nŠ are A½nŠ 2 fþ1; À1g, where these values are chosen with equal probability and independently of each other. For the signal waveform pðtÞ we take a rectangular pulse of width T, being the bit time. For the autocorrelation of the data sequence it is found that R½0Š ¼ 12 1 2 þ ðÀ1Þ2 1 2 ¼ 1 R½mŠ ¼ 12 1 4 þ ðÀ1Þ2 1 4 þ ð1ÞðÀ1Þ14 þ ðÀ1Þð1Þ14 ¼ 0; for m 6¼ 0 ð4:62Þ ð4:63Þ Substituting these values into Equation (4.61) gives the power spectral density of the polar NRZ data signal: SXX ð!Þ ¼  sin ! T ! T 2 T 2 2 ð4:64Þ 80 LINEAR FILTERING OF STOCHASTIC PROCESSES SXX (ω) T –6π/T –4π/T –2π/T 0 2π/T 4π/T 6π/T ω Figure 4.7 The power spectral density of the polar NRZ data signal since the Fourier transform of the rectangular pulse is the well-known sinc function (see Appendix G). The resulting spectrum is shown in Figure 4.7. The disadvantage of the polar NRZ signal is that it has a large value of its power spectrum near the d.c. component, although it does not comprise a d.c. component. On the other hand, the signal is easy to generate, and since it is a simplex signal (see Appendix A) it is power efficient. & Example 4.5: In this example we consider the so-called unipolar RZ (return to zero) data signal. This once more reflects the behaviour of the data pulses. For this signal format the values of A½nŠ are chosen from the set A½nŠ 2 f1; 0g with equal probability and mutually independent. The signalling waveform pðtÞ is defined by & pðtÞ ¼4 1; 0; 0 t < T=2 T=2 t < T ð4:65Þ It is easy to verify that the autocorrelation of the data sequence reads ( R½mŠ ¼ 1 2 ; 1 4 ; m¼0 m 6¼ 0 ð4:66Þ Inserting this result into Equation (4.61) reveals that we end up with an infinite sum of complex exponentials: SXX ð!Þ ¼ T 4  sin ! ! T 4 T 4 2"1 4 þ 1 4 X 1 m¼À1 # expðj!mT Þ ð4:67Þ SPECTRUM OF A RANDOM DATA SIGNAL 81 SXX (ω) T 16 –8π/T –6π/T –4π/T –2π/T 0 2π/T 4π/T 6π/T 8π/T ω Figure 4.8 The power spectral density of the unipolar RZ data signal However, this infinite sum of exponentials can be rewritten as X 1 expðj!mTÞ ¼ 2p X 1    ! À 2pm m¼À1 T m¼À1 T ð4:68Þ which is known as the Poisson sum formula [7]. Applying this sum formula to Equation (4.67) yields SXX ð!Þ ¼ T 16  sin ! ! T 4 T2" 41 þ 2p T X 1 m¼À1  ! À # 2m T ð4:69Þ This spectrum has been depicted in Figure 4.8. Comparing this figure with that of Figure 4.7 a few remarks need to be made. First of all, the first null bandwidth of the RZ signal increased by a factor of two compared to the NRZ signal. This is due to the fact that the pulse width was reduced by the same factor. Secondly, a series of  functions appears in the spectrum. This is due to the fact that the unipolar signal has no zero mean. Besides the large value of the spectrum near zero frequency there is a d.c. component. This is also discovered from the  function at zero frequency. The weights of the  functions scale with the sinc function (Equation (4.69)) and vanish at all zero-crossings of the latter. & Theorem 8 is a powerful tool for calculating the spectrum of all types of formats for data signals. For more spectra of data signals see reference [6]. 82 LINEAR FILTERING OF STOCHASTIC PROCESSES 4.6 PRINCIPLES OF DISCRETE-TIME SIGNALS AND SYSTEMS The description of discrete-time signals and systems follows in a straightforward manner by sampling the functions that describe their continuous counterparts. Especially for the theory and conversion it is most convenient to confine the procedure to ideal sampling, i.e. by multiplying the continuous functions by a sequence of  functions, where these  functions are equally spaced in time. First of all we define the discrete-time  function as follows: & ½nŠ ¼4 1; 0; n ¼ 0; n 6¼ 0 ð4:70Þ or in general & ½n À mŠ ¼4 1; 0; n¼m n 6¼ m ð4:71Þ Based on this latter definition each arbitrary signal x½nŠ can alternatively be denoted as X 1 x½nŠ ¼ x½mŠ ½n À mŠ m¼À1 ð4:72Þ We will limit our treatment to linear time-invariant systems. For discrete-time systems we introduce a similar definition for linear time-invariant systems as we did for continuous systems, namely if : then : X xi½nŠ ) X yi½nŠ aixi½n À miŠ ) aiyi½n À miŠ i i ð4:73Þ for any arbitrary set of constants ai and mi. Also, the convolution follows immediately from the continuous case X 1 X 1 y½nŠ ¼ x½mŠ h½n À mŠ ¼ h½nŠ x½n À mŠ ¼ x½nŠ à h½nŠ m¼À1 m¼À1 ð4:74Þ This latter description will be used for the output y½nŠ of a discrete-time system with the impulse response h½nŠ, which has x½nŠ as an input signal. This expression is directly deduced by sampling Equation (4.12), but can alternatively be derived from Equation (4.72) and the properties of linear time-invariant systems. In many practical situations the impulse response function h½nŠ will have a finite extent. Such filters are called finite impulse response filters, abbreviated as FIR filters, whereas the infinite impulse response filter is abbreviated to the IIR filter. 4.6.1 The Discrete Fourier Transform For continuous signals and systems we have a dual frequency domain description. Now we look for discrete counterparts for both the Fourier transform and its inverse transform, since PRINCIPLES OF DISCRETE-TIME SIGNALS AND SYSTEMS 83 then these transformations can be performed by a digital processor. When considering for discrete-time signals the presentation by means of a sequence of  functions X 1 X 1 x½nŠ ¼ xðtÞ ðt À nTsÞ ¼ xðnTsÞ ðt À nTsÞ n¼À1 n¼À1 X 1 ¼ x½nŠ ðt À nTsÞ n¼À1 ð4:75Þ where 1=Ts is the sampling rate, the corresponding Fourier transform of the sequence x½nŠ is easily achieved, namely X 1 Xð!Þ ¼ x½nŠ expðÀj!nTsÞ n¼À1 ð4:76Þ Due to the discrete character of the time function its Fourier transform is a periodic function of frequency with period 2p=Ts. The inverse transform is therefore x½nŠ ¼ Ts 2p Z p=Ts Àp=Ts Xð!Þ expðjn!TsÞ d! ð4:77Þ In Equations (4.76) and (4.77) the time domain has been discretized. Therefore, the operations are called the discrete-time Fourier transform (DTFT) and the inverse discretetime Fourier transform (IDTFT), respectively. However, the frequency domain has not yet been discretized in those equations. Let us therefore now introduce a discrete presentation of ! as well. We define the radial frequency step as Á! ¼4 2p=T, where T still has to be determined. Moreover, the number of samples has to be limited to a finite amount, let us say N. In order to arrive at a self-consistent discrete Fourier transform this number has to be the same for both the time and frequency domains. Inserting this into Equation (4.76) gives X½kŠ ¼  Xk  2p T ¼ X NÀ1 x½nŠ n¼0  exp Àjk 2p T  nTs ð4:78Þ If we define the ratio of T and Ts as N ¼4 T=Ts, then X½kŠ ¼ X NÀ1 x½nŠ  exp Àj2p  kn n¼0 N ð4:79Þ Inserting the discrete frequency and limited amount of samples as defined above into Equation (4.77) and approximating the integral by a sum yields x½nŠ ¼ Ts 2p X NÀ1 k¼0 X½kŠ  exp j2p nk T  Ts 2p T ð4:80Þ 84 LINEAR FILTERING OF STOCHASTIC PROCESSES or x½nŠ ¼ 1 X NÀ1 X½kŠ  exp j2p nk N k¼0 N ð4:81Þ The transform given by Equation (4.79) is called the discrete Fourier transform (abbreviated DFT) and Equation (4.81) is called the inverse discrete Fourier transform (IDFT). They are a discrete Fourier pair, i.e. inserting a sequence into Equation (4.79) and in turn inserting the outcome into Equation (4.81) results in the original sequence, and the other way around. In this sense the deduced set of two equations is self-consistent. It follows from the derivations that they are used as discrete approximations of the Fourier transforms. Modern mathematical software packages such as Matlab comprise special routines that perform the DFT and IDFT. The algorithms used in these packages are called the fast Fourier transform (FFT). The FFT algorithm and its inverse (IFFT) are just DFT, respectively IDFT, but are implemented in a very efficient way. The efficiency is maximum when the number of samples N is a power of 2. For more details on DFT and FFT see references [10] and [12]. Example 4.6: It is well known from Fourier theory that the transform of a rectangular pulse in the time domain corresponds to a sinc function in the frequency domain (see Appendices E and G). Let us check this result by applying the FFT algorithm of Matlab to a rectangular pulse. Before programming the pulse, a few peculiarities of the FFT algorithm should be observed. First of all, it is only based on the running variables n and k, both running from 0 to N À 1, which means that no negative values along the x axis can be presented. Here it should be emphasized that in Matlab vectors run from 1 to N, which means that when applying this package the x axis is actually shifted over one position. Another important property is that since the algorithm both requires and produces N data values, Equations (4.79) and (4.81) are periodic functions of respectively k and n, and show a periodicity of N. Thanks to this periodicity the negative argument values can be displayed in the second half of the period. This actually means that in Figure 4.9(a) the rectangular time function is centred about zero. Similarly, the frequency function as displayed in Figure 4.9(b) is actually centred about zero as well. In this figure it has been taken that N ¼ 256. In Figure 4.10 the functions are redrawn with the second half shifted over one period to the left. This results in functions centred about zero in both domains and reflects the well-known result from Fourier theory. It will be clear that the actual time scale and frequency scale in this figure are to be determined based on the actual width of the rectangular pulse. & From a theoretical point of view it is impossible for both the time function and the corresponding frequency function to be of finite extent. However, one can imagine that if one of them is limited the parameters of the DFT are chosen such that the transform is a good approximation. Care should be taken with this, as shown in Figure 4.11. In this figure the rectangular pulse in the time domain has been narrowed, which results in a broader function PRINCIPLES OF DISCRETE-TIME SIGNALS AND SYSTEMS 85 1 x [n ] 0 0 x [k ] 50 100 150 200 250 n (a) N =256 0 0 50 100 150 200 250 k (b) Figure 4.9 (a) The FFT applied to a rectangular pulse in the time domain; (b) the transform in the frequency domain 1 x (t ) 0 X (ω) 0 t (a) 0 0 ω (b) Figure 4.10 (a) The shifted rectangular pulse in the time domain; (b) the shifted transform in the frequency domain 86 LINEAR FILTERING OF STOCHASTIC PROCESSES 1 x [n ] 0 X [k ] 50 100 150 200 250 n (a) N =256 0 0 50 100 150 200 250 k (b) Figure 4.11 (a) Narrowed pulse in the time domain; (b) the FFT result, showing aliasing in the frequency domain. In this case two adjacent periods in the frequency domain overlap; this is called aliasing. Although the two functions of Figure 4.11 form an FFT pair, the frequency domain result from Figure 4.11(b) shows a considerable distortion compared to the Fourier transform, which is rather well approximated by Figure 4.9(b). It will be clear that this aliasing can in such cases be prevented by increasing the number of samples N. This has to be done in this case by keeping the number of samples of value 1 the same, and inserting extra zeros in the middle of the interval. 4.6.2 The z-Transform An alternative approach to deal with discrete-time signals and systems is by setting expðj!TsÞ ¼4 z in Equation (4.76). This results in the so-called z-transform, which is defined as X 1 X~ðzÞ ¼4 x½nŠ zÀn n¼À1 ð4:82Þ Comparing this with Equation (4.76) it is concluded that X~ðexpðj!TsÞÞ ¼ Xð!Þ ð4:83Þ Since Equation (4.82) is exactly the same as the discrete Fourier transform, only a different notation has been introduced, the same operations used with the Fourier transform are PRINCIPLES OF DISCRETE-TIME SIGNALS AND SYSTEMS 87 allowed. If we consider Equation (4.82) as the z-transform of the input signal to a linear time-invariant discrete-time system, when calculating the z-transform of the impulse response X 1 H~ ðzÞ ¼ h½nŠ zÀn n¼À1 ð4:84Þ the z-transform of the output is found to be Y~ðzÞ ¼ H~ ðzÞ X~ðzÞ ð4:85Þ A system is called stable if it has a bounded output signal when the input is bounded. A discrete-time system is a stable system if all the poles of H~ ðzÞ lie inside the unit circle of the z plane or in terms of the impulse response [10]: X 1 jh½nŠj < 1 n¼À1 ð4:86Þ The z-transform is a very powerful tool to use when dealing with discrete-time signals and systems. This is due to the simple and compact presentation of it on the one hand and the fact that the coefficients of the different powers zÀn are identified as the time samples at nTs on the other hand. For further details on the z-transform see references [10] and [12]. Example 4.7: Consider a discrete-time system with the impulse response & h½nŠ ¼ an; 0; n!0 n<0 ð4:87Þ and where jaj < 1. The sequence h½nŠ has been depicted in Figure 4.12 for a positive value of a. The z-transform of this impulse response is H~ ðzÞ ¼ 1 þ azÀ1 þ a2zÀ2 þ Á Á Á ¼ 1 1 À azÀ1 ð4:88Þ h [n ] ... n Figure 4.12 The sequence an with 0 < a < 1 88 LINEAR FILTERING OF STOCHASTIC PROCESSES Let us suppose that this filter is excited with an input sequence & x½nŠ ¼ bn; n ! 0 0; n < 0 ð4:89Þ with jbj < 1 and b 6¼ a. Similar to Equation (4.88) the z-transform of this sequence is X~ ðzÞ ¼ 1 þ bzÀ1 þ b2zÀ2 þ Á Á Á ¼ 1 1 À bzÀ1 ð4:90Þ Then the z-transform of the output is Y~ðzÞ ¼ H~ ðzÞ X~ðzÞ ¼ 1 1 À azÀ1 1 1 À bzÀ1 ð4:91Þ The time sequence can be recovered from this by decomposition into partial fractions: 1 1 À azÀ1 1 1 À bzÀ1 ¼ a a À b 1 1 À azÀ1 À a b À b 1 1 À bzÀ1 ð4:92Þ From this the output sequence is easily derived: 8 y½nŠ ¼ < : a aÀ 0; b an À a b À b bn; n!0 n<0 ð4:93Þ & As far as the realization of discrete-time filters is concerned two main types are distinguished, namely the non-recursive filter structure and the recursive. The realization of the non-recursive filter is quite straightforward and the structure is given in Figure 4.13; it is also called the transversal filter or tapped delay line filter. The boxes represent delays of Ts seconds and the outputs are multiplied by an. The delayed and multiplied outputs are added x [n ] Ts Ts Ts a0 a1 a2 aN Σ y [n ] Figure 4.13 The structure of the discrete-time non-recursive filter, transversal filter or tapped delay line filter PRINCIPLES OF DISCRETE-TIME SIGNALS AND SYSTEMS 89 x [n ] + y [n ] Ts Ts Ts -1 bM b2 b1 Σ Figure 4.14 The structure of the discrete-time recursive filter to form the filter output sequence y½nŠ. It is easy to understand that the transfer function of the filter in the z domain is described by the polynomial X n¼N H~ ðzÞ ¼ A~ðzÞ ¼ a0 þ a1zÀ1 þ Á Á Á þ aNzÀN ¼ anzÀn n¼0 ð4:94Þ From the structure it follows that it is a finite impulse response (FIR) filter and there is a simple and direct relation between the multiplication factors and the polynomial coefficients. An FIR filter is inherently stable. The recursive filter is based on a similar tapped delay line filter, which is in a feedback loop depicted in Figure 4.14. The transfer function of the feedback filter is B~ðzÞ ¼ b1zÀ1 þ b2zÀ2 þ Á Á Á þ bMzÀM ð4:95Þ and from the figure it is easily derived that the transfer function of the recursive filter is H~ ðzÞ ¼4 Y~ðzÞ X~ ðzÞ ¼ 1 1 þ B~ðzÞ ð4:96Þ As a rule this transfer function has an infinite impulse response, so it is an IIR filter. The stability of the recursive filter is guaranteed if the denominator polynomial 1 þ B~ðzÞ in Equation (4.96) has no zeros outside the unit circle. In filter synthesis the transfer function is often specified by means of a rational function; i.e. it is given as H~ ðzÞ ¼ a0 þ a1zÀ1 þ a2zÀ2 þ Á Á Á þ aN zÀN 1 þ b1zÀ1 þ b2zÀ2 þ Á Á Á þ bMzÀM ¼ 1 A~ðzÞ þ B~ðzÞ ð4:97Þ It can be seen that this function is realizable as the cascade of the FIR filter from Figure 4.13 and the IIR filter from Figure 4.14, as follows from Equations (4.94) to (4.96), while the same stability criterion is valid as for the IIR filter. The Signal Processing Toolbox from Matlab comprises several commands for the analysis and design of discrete-time filters, one of which is filter, which calculates the output of filters described by Equation (4.97) when excited by a specified input sequence. 90 LINEAR FILTERING OF STOCHASTIC PROCESSES 4.7 DISCRETE-TIME FILTERING OF RANDOM SEQUENCES 4.7.1 Time Domain Description of the Filtering The filtering of a discrete-time stochastic process X½nŠ by a discrete-time linear timeinvariant system with the impulse response h½nŠ is described by the convolution (see Equation (4.74)) X 1 X 1 Y½nŠ ¼ X½mŠ h½n À mŠ ¼ X½n À mŠ h½mŠ ¼ X½nŠ à h½nŠ m¼À1 m¼À1 ð4:98Þ It should be remembered that treatment is confined to real wide-sense stationary processes. Then the mean value of the output sequence is Âà E Y½nŠ ¼ X 1  à E X½n À mŠ h½mŠ ¼ X X 1 h½mŠ ¼ X H~ ð1Þ m¼À1 m¼À1 ð4:99Þ where H~ ð1Þ is the z-transform of h½nŠ evaluated at z ¼ 1, which follows from its definition (see Equation (4.84)). The autocorrelation sequence of the output process, which can be proved to be wide-sense stationary as well, is  à RYY ½mŠ ¼ E½Y½nŠ Y " n þ mŠ # X 1 X 1 ¼E X½n À kŠ h½kŠ X½n þ m À lŠ h½lŠ k¼À1 l¼À1 X 1 X 1 ¼ E½X½n À kŠ X½n þ m À lŠŠ h½kŠ h½lŠ k¼À1 l¼À1 ¼ RXX½mŠ à h½mŠ à h½ÀmŠ ð4:100Þ The cross-correlation sequence between the input and output becomes  à RXY ½mŠ ¼ E X½nŠ " Y ½n þ mŠ # X ¼ E X½nŠ X½n þ m À lŠ h½lŠ X l à ¼ E X½nŠ X½n þ m À lŠ h½lŠ Xl ¼ RXX½m À lŠ h½lŠ ¼ RXX½mŠ à h½mŠ l ð4:101Þ In a similar way is derived RYX½mŠ ¼ RXX½mŠ à h½ÀmŠ ð4:102Þ DISCRETE-TIME FILTERING OF RANDOM SEQUENCES 91 Moreover, the following relation exists: RYY ½mŠ ¼ RXY ½mŠ à h½ÀmŠ ¼ RYX½mŠ à h½mŠ ð4:103Þ 4.7.2 Frequency Domain Description of the Filtering Once the autocorrelation sequence at the output of the linear time-invariant discrete-time system is known, the spectrum at the output follows from Equation (4.100): SYY ð!Þ ¼ SXXð!Þ Hð!Þ HÃð!Þ ¼ SXXð!Þ jHð!Þj2 ð4:104Þ where the functions of frequency have to be interpreted as in Equation (4.76). Using the z-transform, and assuming h½nŠ to be real, we arrive at ~SYY ðzÞ ¼ ~SXXðzÞ H~ ðzÞ H~ ðzÀ1Þ ð4:105Þ since for a real system HÃð!Þ ¼ HðÀ!Þ and consequently H~ ðzÞ ¼ H~ ðzÀ1Þ. Owing to the discrete-time nature of Y½nŠ its spectrum is periodic. According to Equa- tion (3.66) its power is denoted by PY ¼  EY 2½nŠÃ ¼ RYY ½0Š ¼ ¼ Z Ts p=Ts 2p Ts ZÀpp==TTss 2p X X Àp=Ts SXXð!Þ jHð!Þj2 d! SXXð!Þ jH~ ðexpðj!TsÞÞj2 d! ¼ RXX½k À lŠ h½kŠ h½lŠ kl ð4:106Þ The last line of this equation follows from Equation (4.100). Example 4.8: Let us consider the system with the z-transform H~ ðzÞ ¼ 1 1 À azÀ1 ð4:107Þ with jaj < 1; this is the system introduced in Example 4.7. We assume that the system is driven by the stochastic process X½nŠ with spectral density ~SXXðzÞ ¼ 1 or equivalently RXX½mŠ ¼ ½mŠ; later on we will call such a process white noise. Then the output spectral density according to Equation (4.105) is written as ~SYY ðzÞ ¼ z z À a zÀ1 zÀ1 À a ¼ ðz À z aÞð1 À azÞ ð4:108Þ 92 LINEAR FILTERING OF STOCHASTIC PROCESSES Expanding this expression in partial fractions yields ~SYY ðzÞ ¼ 1 1 À a2 1 1 À az þ 1 1 À a2 azÀ1 1 1 À azÀ1 ð4:109Þ Next we expand in series the fractions with z and zÀ1: ~SYY ðzÞ ¼ 1 1 À a2 À 1 þ az þ a2z2 þ Á ÁÁÁ þ 1 1 À! a2 azÀ1ð1 þ azÀ1 þ a2zÀ2 þ Á Á ÁÞ ¼ 1 1 À a2 X0 X 1 aÀnzÀn þ anzÀn n¼À1 n¼1 ¼ 1 1 À a2 X 1 n¼À1 ajnjzÀn ð4:110Þ The autocorrelation sequence of the output is easily derived from this, namely RYY ½mŠ ¼ 1 1 À a2 ajmj ð4:111Þ and in turn it follows immediately that PY ¼ RYY ½0Š ¼ 1 1 À a2 ð4:112Þ The power spectrum is deduced from Equation (4.109), which for that purpose is rewritten as ~SYY ðzÞ ¼ 1 À aðz 1 þ zÀ1Þ þ a2 ð4:113Þ Inserting z ¼ expðj!TsÞ leads to the spectrum in the frequency domain SYY ð!Þ ¼ ~SYY ðexpðj!TsÞÞ ¼ 1 À 2a 1 cosð!TsÞ þ a2 ð4:114Þ In Figure 4.15 this spectrum has been depicted for a ¼ 0:8 and a sampling interval time of 1 second. As a consequence the spectrum has a periodicity of 2 in the angular frequency domain. Clearly, this is a lowpass spectrum. & From this example it is concluded that the z-transform is a powerful tool for analysing discrete-time systems that are driven by a stochastic process. The transformation and its inverse are quite simple and the operations in the z domain are simply algebraic manipulations. Moreover, conversion from the z domain to the frequency domain is just a simple substitution. Matlab comprises the procedures conv and deconv for multiplication and division of polynomials, respectively. The filter command in the Signal Processing Toolbox can do the same job (see also the end of Subsection 4.6.2). Moreover, the same toolbox comprises the command freqz, which, apart from the filter operation, also converts the result to the frequency domain. SYY (ω) 30 SUMMARY 93 25 20 15 10 5 0 –10 –8 –6 –4 –2 0 2 4 6 8 10 ω Figure 4.15 The periodic spectrum of the example with a ¼ 0:8 4.8 SUMMARY The input of a linear time-invariant system is excited with a wide-sense stationary process. In the time domain the output process is described as the convolution of the input process and the impulse response of the system, e.g. a filter. Next from this description such characteristics as mean value and correlation functions are to be determined, using the definitions given in Chapter 2. Although the expression for the autocorrelation function of the output process looks rather complicated, namely a twofold convolution, for certain processes this may result in a tractable description. Applying the Fourier transform and arriving at the power spectral density produces a much simpler expression. The probability density function of the output process can, in general, not be easily determined in analytical form from the input probability density function. There is an important exception for this; namely if the input process has a Gaussian probability density function then the output process also has a Gaussian probability density function. The concept of equivalent noise bandwidth has been defined in order to arrive at an even more simple description of noise filtering in the frequency domain. The theory of noise filtering is applied to a specific stochastic process in order to describe the autocorrelation function and spectrum of random data signals. Next, attention is paid to discrete-time signals and systems. Specials tools for that are dealt with. The discrete Fourier transform (DFT) and its inverse (IDFT) are derived and it is shown that this transform can serve as an approximation of the Fourier transform. Problems when applying the DFT for that purpose are indicated, including the ways used to avoid them. A closely related transform, namely the z-transform, appears to be more tractable for practical applications. The relation to the Fourier transform is quite simple. Finally, it is shown how to apply these transforms to filtering discrete-time processes by discrete-time systems. 94 LINEAR FILTERING OF STOCHASTIC PROCESSES L input R output Figure 4.16 4.9 PROBLEMS 4.1 Consider the network given in Figure 4.16. (a) Calculate the voltage transfer function. Make a plot of its absolute value on a double logarithmic scale using Matlab and with R=L ¼ 1. (b) Calculate the impulse response. Make a plot of it using Matlab. (c) Calculate the response of the network to a rectangular input pulse that starts at t ¼ 0, has height 1 and lasts for 2 seconds. Make a plot of the response using Matlab. 4.2 Derive the Fourier transform of the function f ðtÞ ¼ expðÀ jtjÞ cosð!0tÞ Use Matlab to produce a plot of the transform with ¼ 1 and !0 ¼ 10. 4.3 Consider the circuit given in Figure 4.17. (a) Determine the impulse response of the circuit. Make a sketch of it. (b) Calculate the transfer function Hð!Þ. (c) Determine and draw the response yðtÞ to the input signal xðtÞ ¼ A rect ½ðt À 1 2 T Þ=T Š, where A is a constant. (See Appendix E for the definition of the rectangular pulse function rectðÁÞÞ. (d) Determine and draw the response yðtÞ to the input signal xðtÞ ¼ A rect ½ðt À TÞ=ð2Tފ, where A is a constant. 4.4 A stochastic process XðtÞ ¼ A sinð!0t À ÂÞ is given, with A and !0 real, positive constants and  a random variable that is uniformly distributed on the interval ð0; 2pŠ. This process is applied to a linear time-invariant network with the impulse response + x (t ) + (.)dt y (t ) delay - T Figure 4.17 PROBLEMS 95 hðtÞ ¼ uðtÞ expðÀt=0Þ. (The unit-step function uðtÞ is defined in Appendix E.) Here 0 is a positive, real constant. Derive an expression for the output process. 4.5 A wide-sense stationary Gaussian process with spectral density N0=2 is applied to the input of a linear time-invariant filter. The impulse response of the filter is & hðtÞ ¼ 1; 0 < t < T 0; elsewhere (a) Sketch the impulse response of the filter. (b) Calculate the mean value and the variance of the output process. (c) Determine the probability density function of the output process. (d) Calculate the autocorrelation function of the output process. (e) Calculate the power spectrum of the output process. 4.6 A wide-sense process XðtÞ has the autocorrelation function RXXðÞ ¼ A2þ Bð1 À jj=TÞ for jj < T and with A, B and T positive constants. This process is used as the input to a linear, time-invariant system with the impulse response hðtÞ ¼ uðtÞ À uðt À TÞ. (a) Sketch RXXðÞ and hðtÞ. (b) Calculate the mean value of the output process. 4.7 White noise with spectral density of N0=2 V2=Hz is applied to the input of the system given in Problem 4.1. (a) Calculate the spectral density of the output. (b) Calculate the mean quadratic value of the output process. 4.8 Consider the circuit in Figure 4.18, where XðtÞ is a wide-sense stationary voltage process. Measurements on the output voltage process YðtÞ reveal that this process is Gaussian. Moreover, it is measured as RYY ðÞ ¼ 9 expðÀ jjÞ þ 25; where ¼ 1 RC (a) Determine the probability density function fY ðyÞ. Plot this function using Matlab. (b) Calculate and sketch the spectrum SXXð!Þ of the input process. (c) Calculate and sketch the autocorrelation function RXXðÞ of the input process. R X (t ) C Y (t ) Figure 4.18 96 LINEAR FILTERING OF STOCHASTIC PROCESSES 4.9 Two linear time-invariant systems have impulse responses of h1ðtÞ and h2ðtÞ, respectively. The process X1ðtÞ is applied to the first system and the corresponding response reads Y1ðtÞ. Similarly, the input process X2ðtÞ to the second system results in the response Y2ðtÞ. Calculate the cross-correlation function of Y1ðtÞ and Y2ðtÞ in terms of h1ðtÞ, h2ðtÞ and the cross-correlation function of X1ðtÞ and X2ðtÞ, assuming X1ðtÞ and X2ðtÞ to be jointly wide-sense stationary. 4.10 Two systems are cascaded. A stochastic process XðtÞ is applied to the input of the first system having the impulse response h1ðtÞ. The response of this system is WðtÞ and serves as the input of the second system with the impulse response h2ðtÞ. The response of this second system reads YðtÞ. Calculate the cross-correlation function of WðtÞ and YðtÞ in terms of h1ðtÞ and h2ðtÞ, and the autocorrelation function of XðtÞ. Assume that XðtÞ is wide-sense stationary. 4.11 The process (called the signal) XðtÞ ¼ cosð!0t À ÂÞ, with  uniformly distributed on the interval ð0; 2pŠ, is added to white noise with spectral density N0=2. The sum is applied to the RC network given in Figure 4.2. (a) Calculate the power spectral densities of the output signal and the output noise. (b) Calculate the ratio of the mean output signal power and the mean output noise power, the so-called signal-to-noise ratio. (c) For what value of 0 ¼ RC does this ratio become maximal? 4.12 White noise with spectral density of N0=2 is applied to the input of a linear timeinvariant system with the transfer function Hð!Þ ¼ ð1 þ j!0ÞÀ2. Calculate the power of the output process. 4.13 A differentiating network may be considered as a linear time-invariant system with the transfer function Hð!Þ ¼ j!. If the input is a wide-sense stationary process XðtÞ, then the output process is dXðtÞ=dt ¼ X_ ðtÞ. Show that: (a) RXX_ ð Þ ¼ dRXX ð d Þ (b) RX_ X_ ð Þ ¼ À d2RXXð Þ d 2 4.14 To the differentiating network presented in Problem 4.13 a wide-sense stationary process XðtÞ is applied. The corresponding output process is YðtÞ. (a) Are the random variables XðtÞ and YðtÞ, both considered at the same fixed time t, orthogonal? (b) Are these random variables uncorrelated? 4.15 A stochastic process with the power spectral density of SXX ð!Þ ¼ ð1 1 þ !2Þ2 is applied to a differentiating network. (a) Find the power spectral density of the output process. (b) Calculate the power of the derivative of XðtÞ. PROBLEMS 97 4.16 Consider the stochastic process Z tþT YðtÞ ¼ XðÞ d tÀT where XðtÞ is a wide-sense stationary process. (a) Design a linear time-invariant system that produces the given relation between its input process XðtÞ and the corresponding output process YðtÞ. (b) Express the autocorrelation function of YðtÞ in terms of that for XðtÞ. (c) Express the power spectrum of YðtÞ in terms of that for XðtÞ. 4.17 Reconsider Problem 4.16. (a) Prove that the given integration can be realized by a linear time-invariant filter with an impulse response rect½t=ð2Tފ: (b) Fill an array in Matlab with a harmonic function, let us say a cosine. Take, for example, 10 cycles of 1=ð2pÞ Hz and increments of 0.01 for the time parameter. Plot the function. (c) Generate a Gaussian noise wave with mean of zero and unit variance, and of the same length as the cosine using the Matlab command randn, add this wave to the cosine and plot the result. Note that each time you run the program a different wave is produced. (d) Program a vector consisting of all ones. Convolve this vector with the cosine plus noise vector and observe the result. Take different lengths, e.g. equivalent to 2T ¼ 0:05, 0.10 and 0.20. Explain the differences in the different curves. 4.18 In FM detection the white additive noise is converted into noise with spectral density SNN ð!Þ ¼ ! 2 and assume that this noise is wide-sense stationary. Suppose that the signal spectrum is 8 SXX ð!Þ ¼ < : S0 ; 2 0; j!j W j!j > W and that in the receiver the detected signal is filtered by an ideal lowpass filter of bandwidth W. (a) Calculate the signal-to-noise ratio. In audio FM signals so-called pre-emphasis and de-emphasis filtering is applied to improve the signal-to-noise ratio. To that end prior to modulation and transmission the audio baseband signal is filtered by the pre-emphasis filter with the transfer function Hð!Þ ¼ 1 þ j ! ; Wp Wp < W 98 LINEAR FILTERING OF STOCHASTIC PROCESSES At the receiver side the baseband signal is filtered by the de-emphasis filter such that the spectrum is once more flat and equal to S0=2. (b) Make a block schematic of the total communication scheme. (c) Sketch the different signal and noise spectra. (d) Calculate the improvement factor of the signal-to-noise ratio. (e) Evaluate the improvement in dB for the practical values: W=ð2pÞ ¼ 15 kHz, Wp=ð2Þ ¼ 2:1 kHz. 4.19 A so-called nth-order Butterworth filter is defined by the squared value of the amplitude of the transfer function jHð!Þj2 ¼ 1 þ 1 ð!=W Þ2n where n is an integer, which is called the order of the filter. W is the À3 dB bandwidth in radians per second. (a) Use Matlab to produce a set of curves that present this squared transfer as a function of frequency; plot the curves on a double logarithmic scale for n ¼ 1; 2; 3; 4. (b) Calculate and evaluate the equivalent noise bandwidth for n ¼ 1 and n ¼ 2. 4.20 For the transfer function of a bandpass filter it is given that jHð!Þj2 ¼ 1 þ 1 ð! À !0Þ2 þ 1 þ 1 ð! þ !0Þ2 (a) Use Matlab to plot jHð!Þj2 for !0 ¼ 10. (b) Calculate the equivalent noise bandwidth of the filter. (c) Calculate the output noise power when wide-sense stationary noise with spectral density of N0=2 is applied to the input of this filter. 4.21 Consider the so-called Manchester (or split-phase) signalling format defined by & 1; 0 t < T=2 pðtÞ ¼ À1; T=2 t < T where T is the bit time. The data symbols A½nŠ are selected from the set f1; À1g with equal probability and are mutually independent. (a) Sketch the Manchester coded signal of the sequence 1010111001. (b) Calculate the power spectral density of this data signal. Use Matlab to plot it. (c) Discuss the properties of the spectrum in comparison to the polar NRZ signal. 4.22 In the bipolar NRZ signalling format the binary 1’s are alternately mapped to A½nŠ ¼ þ1 volt and A½nŠ ¼ À1volt. The binary 0 is mapped to A½nŠ ¼ 0 volt. The bits are selected with equal probability and are mutually independent. (a) Sketch the bipolar NRZ coded signal of the sequence 1010111001. PROBLEMS 99 (b) Calculate the power spectral density of this data signal. Use Matlab to plot it. (c) Discuss the properties of the spectrum in comparison to the polar NRZ signal. 4.23 Reconsider Example 4.6. Using Matlab fill in an array of size 256 with a rectangular function of width 50. Apply the FFT procedure to that. Square the resulting array and subsequently apply the IFFT procedure. (a) Check the FFT result for aliasing. (b) What in the time domain is the equivalence of squaring in the frequency domain? (c) Check the IFFT result with respect to your answer to (b). Now fill another array of size 256 with a rectangular function of width 4 and apply the FFT to it. (d) Check the result for aliasing. (e) Multiply the FFT result of the 50 wide pulse width that of the 4 wide pulse and IFFT the multiplication. Is the result what you expected in view of the result from (d)? 4.24 In digital communications a well-known disturbance of the received data symbols is the so-called intersymbol interference (see references [6], [9] and [11]). It is actually the spill-over of the pulse representing a certain bit to the time interval assigned to the adjacent pulses that represent different bits. This disturbance is a consequence of the distortion in the transmission channel. By means of proper filtering, called equalization, the intersymbol interference can be removed or minimized. Assume that each received pulse that represents a bit is sampled once and that the sampled sequence is represented by its z-transform R~ðzÞ. For an ideal channel, i.e. a channel that does not produce intersymbol interference, we have R~ðzÞ ¼ 1. Let us now consider a channel with intersymbol interference and design a discretetime filter that equalizes the channel. If the z-transform of the filter impulse response is denoted by F~ðzÞ, then for the equalized pulse the condition R~ðzÞ F~ðzÞ ¼ 1 should be satisfied. Therefore, in this problem the sequence R~ðzÞ is known and the sequence F~ðzÞ has to be solved to satisfy this condition. It appears that the Matlab command deconv is not well suited to solving this problem. (a) Suppose that F~ðzÞ comprises three terms. Show that the condition for equalization is equivalent to 2 r½0Š 4 r½1Š r½2Š r½À1Š r½0Š r½1Š 32 3 23 r½À2Š f ½À1Š 0 r½À1Š 5 Á 4 f ½0Š 5 ¼ 4 1 5 r½0Š f ½1Š 0 (b) Consider a received pulse R~ðzÞ ¼ 0:1z þ 1 À 0:2zÀ1 þ 0:1zÀ2. Design the equalizer filter of length 3. (c) As the quality factor with respect to intersymbol interference we define the ‘worst case interference’. It is the sum of the absolute signal samples minus the desired sample value 1. Calculate the output sequence of the equalizer designed in (b) using conv and calculate its worst-case interference. Compare this with the unequalized worst-case interference. 100 LINEAR FILTERING OF STOCHASTIC PROCESSES (d) Redo the equalizer design for filter lengths 5 and 7, and observe the change in the worst-case interference. 4.25 Find the transfer function and filter structure of the discrete-time system when the following relations exist between the input and output: (a) y½nŠ þ 2y½n À 1Š þ 0:5y½n À 2Š ¼ x½nŠ À x½n À 2Š (b) 4y½nŠ þ y½n À 1Š À 2y½n À 2Š À 2y½n À 3Š ¼ x½nŠ þ x½n À 1Š À x½n À 2Š (c) Are the given systems stable? Hint: use the Matlab command roots to compute the roots of polynomials. 4.26 White noise with spectral density of N0=2 is applied to an ideal lowpass filter with bandwidth W. (a) Calculate the autocorrelation function of the output process. Use Matlab to plot this function. (b) The output noise is sampled at the time instants tn ¼ np=W with n integer. What can be remarked with respect to the sample values? 4.27 A discrete-time system has the transfer function H~ ðzÞ ¼ 1 þ 0:9zÀ1 þ 0:7zÀ2. To the input of the system the signal with z-transform X~ðzÞ ¼ 0:7 þ 0:9zÀ1 þ zÀ2 is applied. This signal is disturbed by a wide-sense stationary white noise sequence. The autocorrelation sequence of this noise is RNN½mŠ ¼ 0:01 ½mŠ. (a) Calculate the signal output sequence. (b) Calculate the autocorrelation sequence at the output. (c) Calculate the maximum value of the signal-to-noise ratio. At what moment in time will that occur? Hint: you can eventually use the Matlab command conv to perform the required polynomial multiplications. In this way the solution found using pencil and paper can be checked. 4.28 The transfer function of a discrete-time filter is given by H~ ðzÞ ¼ 1 À 1 0:8zÀ1 (a) Use Matlab’s freqz to plot the absolute value of the transfer function in the frequency domain. (b) If the discrete-time system operates at a sampling rate of 1 MHz and a sine wave of 50 kHz and an amplitude of unity is applied to the filter input, compute the power of the corresponding output signal. (c) A zero mean white Gaussian noise wave is added to the sine wave at the input such that the signal-to-noise ratio amounts to 0 dB. Compute the signal-to-noise ratio at the output. (d) Use the Matlab command randn to generate the noise wave. Design and implement a procedure to test whether the generated noise wave is indeed approximately white noise. (e) Check the analytical result of (c) by means of proper operations on the waves that are generated by Matlab. 5 Bandpass Processes Bandpass processes often occur in electrical engineering, mostly as a result of the bandpass filtering of white noise. This is due to the fact that in electrical engineering in general, and specifically in telecommunications, use is made of modulation of signals. These information-carrying signals have to be filtered in systems such as receivers to separate them from other, unwanted signals in order to enhance the quality of the wanted signals and to prepare them for further processing such as detection. Before dealing with bandpass processes we will present a summary of the description of deterministic bandpass signals. 5.1 DESCRIPTION OF DETERMINISTIC BANDPASS SIGNALS There are many reasons why signals are modulated. Doing so shifts the spectrum to a certain frequency, so that a bandpass signal results (see Section 3.4). On processing, for example, reception of a telecommunication signal, such signals are bandpass filtered in order to separate them in frequency from other (unwanted) signals and to limit the amount of noise power. We consider signals that consist of a high-frequency carrier modulated in amplitude or phase by a time function that varies much more slowly than the carrier. For instance, amplitude modulation (AM) signals are written as sðtÞ ¼ Að1 þ mðtÞÞ cos !0t ð5:1Þ where A is the amplitude of the unmodulated carrier, mðtÞ is the low-frequency modulating signal and !0 is the carrier angular frequency. Note that lower case characters are used here, since in this section we discuss deterministic signals. Assuming that ð1 þ mðtÞÞ is never negative, then sðtÞ looks like a harmonic signal whose amplitude varies with the modulating signal. A frequency-modulated signal is written as  Zt  sðtÞ ¼ A cos !0t þ ðÞ d 0 ð5:2Þ Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 102 BANDPASS PROCESSES The instantaneous angular frequency of this signal is !0 þ ðtÞ and is found by differentiating the argument of the cosine with respect to t. In this case the slowly varying function ðtÞ carries the information to be transmitted. The frequency-modulated signal has a constant amplitude, but the zero crossings will change with the modulating signal. The most general form of a modulated signal is given by sðtÞ ¼ aðtÞ cos½!0t þ ðtފ ð5:3Þ In this equation aðtÞ is the amplitude modulation and ðtÞ the phase modulation, while the derivative dðtÞ=dt represents the frequency modulation of the signal. Expanding the cosine of Equation (5.3) yields sðtÞ ¼ aðtÞ½cos ’ðtÞ cos !0t À sin ’ðtÞ sin !0tŠ ¼ xðtÞ cos !0t À yðtÞ sin !0t ð5:4Þ with xðtÞ ¼4 aðtÞ cos ’ðtÞ yðtÞ ¼4 aðtÞ sin ’ðtÞ ð5:5Þ The functions xðtÞ and yðtÞ are called the quadrature components of the signal. Signal xðtÞ is called the in-phase component or I-component and yðtÞ the quadrature or Q-component. They will vary little during one period of the carrier. Combining the quadrature components to produce a complex function will give a representation of the modulated signal in terms of the complex envelope zðtÞ ¼4 xðtÞ þ jyðtÞ ¼ aðtÞ exp½ j’ðtފ ð5:6Þ When the carrier frequency !0 is known, the signal sðtÞ can unambiguously be recovered from this complex envelope. It is easily verified that sðtÞ ¼ RefzðtÞ expð j!0tÞg ¼ 1 2 ½zðtÞ expð j!0tÞ þ zÃðtÞ expðÀj!0tފ ð5:7Þ where RefÁg denotes the real part of the quantity in the braces. Together with the carrier frequency !0, the signal zðtÞ constitutes an alternative and complete description of the modulated signal. The expression zðtÞ expðj!0tÞ is called the analytic signal or pre-envelope. The complex function zðtÞ can be regarded as a phasor in the xy plane. The end of the phasor moves around in the complex plane, while the plane itself rotates with an angular frequency of !0 and the signal sðtÞ is the projection of the rotating phasor on a fixed line. If the movement of the phasor zðtÞ with respect to the rotating plane is much slower than the speed of rotation of the plane, the signal is quasi-harmonic. The phasor in the complex z plane has been depicted in Figure 5.1. It is stressed that zðtÞ is not a physical signal but a mathematically defined auxiliary signal to facilitate the calculations. The name complex envelope suggests that there is a relationship with the envelope of a modulated signal. This envelope is interpreted as DESCRIPTION OF DETERMINISTIC BANDPASS SIGNALS 103 Im {z(t )} ω0t y (t ) a(t ) φ(t ) x (t ) Re {z (t )} Figure 5.1 The phasor of sðtÞ in the complex z plane the instantaneous amplitude of the signal, in this case aðtÞ. Now the relationship is clear, namely pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi aðtÞ ¼ jzðtÞj ¼ x2ðtÞ þ y2ðtÞ ’ðtÞ ¼ arg½zðtފ ¼ arctan yðtÞ xðtÞ ð5:8Þ It is concluded that the complex envelope, in contrast to the envelope, not only comprises information about the envelope aðtÞ but also about the phase ’ðtÞ. As far as detection is concerned, the quadrature component xðtÞ is restored by multiplying sðtÞ by cos !0t and removing the double frequency components with a lowpass filter: sðtÞ cos !0t ¼ 1 2 xðtÞð1 þ cos 2!0tÞ À 1 2 yðtÞ sin 2!0t ð5:9Þ This multiplication operation can be performed by the modulator scheme presented in Figure 3.5. After lowpass filtering, the signal produced will be xðtÞ=2. The second quadrature component yðtÞ is restored in a similar way, by multiplying sðtÞ by À sin !0t and using a lowpass filter to remove the double frequency components. A circuit that delivers an output signal that is a function of the amplitude modulation is called a rectifier and such a circuit will always involve a nonlinear operation. The quadratic rectifier is a typical rectifier; it has an output signal in proportion to the square of the envelope. This output is achieved by squaring the signal and reads s2ðtÞ ¼ 12½x2ðtÞ þ y2ðtފ þ 1 2 ½x2ðtÞ À y2ðtފ cos 2!0t À xðtÞyðtÞ sin 2!0t ð5:10Þ By means of a lowpass filter the frequency terms in the vicinity of 2!0 are removed, so that the output is proportional to jzðtÞj2 ¼ a2ðtÞ ¼ x2ðtÞ þ y2ðtÞ. A linear rectifier, which may consist of a diode and a lowpass filter, yields aðtÞ. A circuit giving an output signal that is proportional to the instantaneous frequency deviation ’0ðtÞ is known as a discriminator. Its output is proportional to d½Imfln zðtÞgŠ=dt. 104 BANDPASS PROCESSES If the signal sðtÞ comprises a finite amount of energy then its Fourier transform exists. Using Equation (5.7) the spectrum of this signal is found to be Z1 Sð!Þ ¼ 1 2 ½zðtÞ expðj!0tÞ þ zÃðtÞ expðÀj!0tފ expðÀj!tÞ dt À1 ¼ 12½Zð! À !0Þ þ ZÃðÀ! À !0ފ ð5:11Þ where Sð!Þ and Zð!Þ are the signal spectra (or Fourier transform) of sðtÞ and zðtÞ, respectively, and * denotes the complex conjugate. The quadrature components and zðtÞ vary much more slowly than the carrier and will be baseband signals. The modulus of the spectrum of the signal, jSð!Þj, has two narrow peaks, one at the frequency !0 and the other at À!0. Consequently, sðtÞ is called a narrowband signal. The spectrum of Equation (5.11) is Hermitian, i.e. SðÀ!Þ ¼ SÃð!Þ, a condition that is imposed by the fact that sðtÞ is real. Quasi-harmonic signals are often filtered by bandpass filters, i.e. filters that pass frequency components in the vicinity of the carrier frequency and attenuate other frequency components. The transfer function of such a filter may be written as Hð!Þ ¼ Hlð! À !cÞ þ HlÃðÀ! À !cÞ ð5:12Þ where the function Hlð!Þ is a lowpass filter; it is called the equivalent baseband transfer function. Equation (5.12) is Hermitian, because hðtÞ, being the impulse response of a physical system, is a real function. However, the equivalent baseband function Hlð!Þ will not be Hermitian in general. Note the similarity of Equations (5.12) and (5.11). The only difference is the carrier frequency !0 and the characteristic frequency !c. For !c, an arbitrary frequency in the passband of Hð!Þ may be selected. In Equation (5.7) the characteristic frequency !0 need not necessarily be taken as equal to the oscillator frequency. A shift in the characteristic frequency over Á! ¼ !1 merely introduces a factor of expðÀj!1tÞ in the complex envelope: zðtÞ expðj!0tÞ ¼ ½zðtÞ expðÀj!1tފ exp½jð!0 þ !1ÞtŠ ð5:13Þ This shift does not change the signal sðtÞ. From this it will be clear that the complex envelope is connected to a specific characteristic frequency; when this frequency changes the complex envelope will change as well. A properly selected characteristic frequency, however, can simplify calculations to a large extent. Therefore, it is important to take the characteristic frequency equal to the oscillator frequency. Moreover, we select the characteristic frequency of the filter equal to that value, i.e. !c ¼ !0. Let us suppose that a modulated signal is applied to the input of a bandpass filter. It appears that using the concepts of the complex envelope and equivalent baseband transfer function the output signal is easily described, as will follow from the sequel. The signal spectra of input and output signals are denoted by Sið!Þ and Soð!Þ, respectively. It then follows that Soð!Þ ¼ Sið!Þ Hð!Þ ð5:14Þ DESCRIPTION OF DETERMINISTIC BANDPASS SIGNALS 105 Zi*(−ω−ω0) Zi(ω−ω0) Hl*(−ω−ω0) Hl(ω−ω0) −ω0 0 ω0 ω Figure 5.2 A sketch of the cross-terms from the right-hand member of Equation (5.15) Invoking Equations (5.11) and (5.12) this can be rewritten as Soð!Þ ¼ 12½Zoð! À !0Þ þ ZoÃðÀ! À !0ފ ¼ 12½Zið! À !0Þ þ ZiÃðÀ! À !0ފ ½Hlð! À !0Þ þ HlÃðÀ! À !0ފ ð5:15Þ where Zið!Þ and Zoð!Þ are the spectra of the complex envelopes of input and output signals, respectively. If it is assumed that the spectrum of the input signal has a bandpass character according to Equations (4.40), (4.41) and (4.42), then the cross-terms Zið! À !0Þ HlÃðÀ! À !0Þ and ZiÃðÀ! À !0Þ Hlð! À !0Þ vanish (see Figure 5.2). Based on this conclusion Equation (5.15) reduces to Zoð! À !0Þ ¼ Zið! À !0Þ Hlð! À !0Þ ð5:16Þ or Zoð!Þ ¼ Zið!Þ Hlð!Þ ð5:17Þ Transforming Equation (5.17) to the time domain yields Z1 zoðtÞ ¼ hlðÞziðt À  Þ d À1 ð5:18Þ with zoðtÞ and ziðtÞ the complex envelopes of the input and output signals, respectively. The function hlðtÞ is the inverse Fourier transform of Hlð!Þ and represents the complex impulse response of the equivalent baseband system, which is defined by means of Hlð!Þ or the dual description hlðtÞ. The construction of the equivalent baseband system Hlð!Þ from the actual system is illustrated in Figure 5.3. This construction is quite simple, namely removing the part of the function around À!0 and shifting the remaining portion around !0 to the baseband, where zero replaces the original position of !0. From this figure it is observed that 106 BANDPASS PROCESSES Hl*(−ω−ω0) Hl(ω−ω0) −ω0 0 ω0 ω Hl(ω) 0 ω Figure 5.3 Construction of the transfer function of the equivalent baseband system in general Hlð!Þ is not Hermitian, and consequently hlðtÞ will not be a real function. This may not surprise us since it is not the impulse response of a real system but an artificially constructed one that only serves as an intermediate to facilitate calculations. From Equations (5.17) and (5.18) it is observed that the relation between the output of a bandpass filter and the input (mostly a modulated signal) is quite simple. The signals are completely determined by their complex envelopes and the characteristic frequency !0. Using the latter two equations the transmission is reduced to the well-known transmission of a baseband signal. After transforming the signal and the filter transfer function to equivalent baseband quantities the relationship between the output and input is greatly simplified, namely a multiplication in the frequency domain or a convolution in the time domain. Once the complex envelope of the output signal is known, the output signal itself is recovered using Equation (5.7). Of course, using the direct method via Equation (5.14) is always allowed and correct, but many times the method based on the equivalent baseband quantities is simpler, for instance in the case after the bandpass filtering envelope detection is applied. This envelope follows immediately from the complex envelope. 5.2 QUADRATURE COMPONENTS OF BANDPASS PROCESSES Analogously to modulated deterministic signals (as described in Section 5.1), stochastic bandpass processes may be described by means of their quadrature components. Consider the process NðtÞ ¼ AðtÞ cos½!0t þ Èðtފ ð5:19Þ with AðtÞ and ÈðtÞ stochastic processes. The quadrature description of this process is readily found by rewriting this latter equation by applying a basic trigonometric relation NðtÞ ¼ AðtÞ cos ÈðtÞ cos !0t À AðtÞ sin ÈðtÞ sin !0t ¼ XðtÞ cos !0t À YðtÞ sin !0t ð5:20Þ QUADRATURE COMPONENTS OF BANDPASS PROCESSES 107 In this expression the quadrature components are defined as XðtÞ ¼4 AðtÞ cos ÈðtÞ YðtÞ ¼4 AðtÞ sin ÈðtÞ ð5:21Þ From these equations the processes describing the amplitude and phase of NðtÞ are easily recovered: pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi AðtÞ ¼4 X2ðtÞ þ Y2ðtÞ  ÈðtÞ ¼4 arctan Y ðtÞ XðtÞ ð5:22Þ The processes XðtÞ and YðtÞ are stochastic lowpass processes (or baseband processes), while Equation (5.20) presents a general description of bandpass processes. In the sequel we will derive relations between these lowpass processes XðtÞ and YðtÞ, and the lowpass processes on the one hand and the bandpass process NðtÞ on the other. Those relations refer to mean values, correlation functions, spectra, etc., i.e. characteristics that have been introduced in previous chapters. Once again we assume that the process NðtÞ is wide-sense stationary with a mean value of zero. A few interesting properties can be stated for such a bandpass process. The properties are given below and proved subsequently. Properties of bandpass processes If NðtÞ is a wide-sense stationary bandpass process with mean value zero and quadrature components XðtÞ and YðtÞ, then XðtÞ and YðtÞ have the following properties: 1. XðtÞ and YðtÞare jointly wide-sense stationary: 2. E½Xðtފ ¼ E½Yðtފ ¼ 0 3. E½X2ðtފ ¼ E½Y2ðtފ ¼ E½N2ðtފ 4. RXXðÞ ¼ RYY ðÞ 5. RYXð Þ ¼ ÀRXY ð Þ 6. RXY ð0Þ ¼ RYXð0Þ ¼ 0 7. SYY ð!Þ ¼ SXXð!Þ 8. SXXð!Þ ¼ LpfSNNð! À !0Þ þ SNN ð! þ !0Þg 9. SXY ð!Þ ¼ j LpfSNNð! À !0Þ À SNNð! þ !0Þg 10. SYXð!Þ ¼ ÀSXY ð!Þ ð5:23Þ ð5:24Þ ð5:25Þ ð5:26Þ ð5:27Þ ð5:28Þ ð5:29Þ ð5:30Þ ð5:31Þ ð5:32Þ In Equations (5.30) and (5.31), LpfÁg denotes the lowpass part of the expression in the braces. 108 BANDPASS PROCESSES Proofs of the properties: Here we shall briefly prove the properties listed above. A few of them will immediately be clear. Expression (5.29) follows directly from Equation (5.26), and Equation (5.32) from Equation (5.27). Moreover, using Equation (2.48), Equation (5.28) is a consequence of Equation (5.27). Invoking the definition of the autocorrelation function, it follows after some manipulation from Equation (5.20) that RNNðt; t þ Þ ¼ 12f½RXXðt; t þ  Þ þ RYY ðt; t þ  ފ cos !0 À ½RXY ðt; t þ  Þ À RYXðt; t þ ފ sin !0 þ ½RXXðt; t þ Þ À RYY ðt; t þ ފ cos !0ð2t þ Þ À ½RXY ðt; t þ  Þ þ RYXðt; t þ ފ sin !0ð2t þ  Þg ð5:33Þ Since we assumed that NðtÞ is a wide-sense stationary process, Equation (5.33) has to be independent of t. Then, from the last term of Equation (5.33) it is concluded that RXY ðt; t þ Þ ¼ ÀRYXðt; t þ Þ ð5:34Þ and from the last but one term of Equation (5.33) RXXðt; t þ  Þ ¼ RYY ðt; t þ Þ ð5:35Þ Using these results it follows from the first two terms of Equation (5.33) that RXXðt; t þ Þ ¼ RXXðÞ ¼ RYY ð Þ ð5:36Þ and RXY ðt; t þ  Þ ¼ RXY ðÞ ¼ ÀRYXðÞ ð5:37Þ thereby establishing properties 4 and 5. Equation (5.33) can now be rewritten as RNN ðÞ ¼ RXXðÞ cos !0 À RXY ð Þ sin !0 ð5:38Þ If we substitute  ¼ 0 in this expression and use Equation (5.26), property 3 follows. The expected value of NðtÞ reads E½Nðtފ ¼ E½Xðtފ cos !0t À E½Yðtފ sin !0t ¼ 0 ð5:39Þ This equation is satisfied only if E½Xðtފ ¼ E½Yðtފ ¼ 0 ð5:40Þ so that now property 2 has been established. However, this means that now property 1 has been proved as well, since the mean values of XðtÞ and YðtÞ are independent of t QUADRATURE COMPONENTS OF BANDPASS PROCESSES 109 (property 2) and also the autocorrelation and cross-correlation functions Equations (5.36) and (5.37). By transforming Equation (5.38) to the frequency domain we arrive at SNN ð!Þ ¼ 1 2 ½SXX ð! À !0Þ þ SXX ð! þ !0ފ þ 1 2 j½SXY ð! À !0Þ À SXY ð! þ !0ފ ð5:41Þ and using this expression gives SNN ð! À !0Þ ¼ 1 2 ½SXX ð! À 2!0Þ þ SXX ð!ފ þ 1 2 j½SXY ð! À 2!0Þ À SXY ð!ފ SNN ð! þ !0Þ ¼ 1 2 ½SXX ð!Þ þ SXX ð! þ 2!0ފ þ 1 2 j½SXY ð!Þ À SXY ð! þ 2!0ފ ð5:42Þ ð5:43Þ Adding Equations (5.42) and (5.43) produces property 8. Subtracting Equation (5.43) from Equation (5.42) produces property 9. Some of those properties are very peculiar; namely the quadrature processes XðtÞ and YðtÞ both have a mean value of zero, are wide-sense stationary, have identical autocorrelation functions and as a consequence have the same spectrum. The processes XðtÞ and YðtÞ comprise the same amount of power and this amount equals that of the original bandpass process NðtÞ (property 3). At first sight this property may surprise us, but after a closer inspection it is recalled that XðtÞ and YðtÞ are the quadrature processes of NðtÞ, and then the property is obvious. Finally, at each moment of time t, the random variables XðtÞ and YðtÞ are orthogonal (property 6). When the spectrum of NðtÞ is symmetrical about !0, the stochastic processes XðtÞ and YðtÞ are orthogonal processes (this follows from property 9 and will be further explained by the next example). In the situation at hand the cross-power spectral density is identical to zero. If, moreover, the processes XðtÞ and YðtÞ are Gaussian, then they are also independent. Example 5.1: In Figure 5.4(a) an example is depicted of a spectrum of a bandpass process. In this figure the position of the characteristic frequency !0 is clearly indicated. On the positive part of the x axis the spectrum covers the region from !0 À W1 to !0 þ W2 and on the negative part of the x axis from À!0 À W2 to À!0 þ W1. Therefore, the bandwidth of the process is W ¼ W1 þ W2. For a bandpass process the requirement W1 < !0 has to be satisfied. In Figure 5.4(b) the spectrum SNNð! À !0Þ is presented; this spectrum is obtained by shifting the spectrum given in Figure 5.4(a) by !0 to the right. Similarly, the spectrum SNNð! þ !0Þ is given in Figure 5.4(c), and this figure is yielded by shifting the spectrum of Figure 5.4(a) by !0 to the left. From Figures 5.4(b) and (c) the spectra of the quadrature components can be constructed using the relations of Equations (5.30) and (5.31). By adding the spectra of Figure 5.4(b) and Figure 5.4(c) the spectra SXXð!Þ and SYY ð!Þ are found (see Figure 5.4(d)). Next, by subtracting the spectra of Figure 5.4(b) and Figure 5.4(c) we arrive at ÀjSXY ð!Þ ¼ jSYXð!Þ (see Figure 5.4(e)). When adding and subtracting as described above, those parts of the spectra that are concentrated about 2!0 and À2!0 have to be ignored. This is in accordance with Equations (5.30) and (5.31). These equations include a lowpass filtering after addition and subtraction, respectively. & 110 BANDPASS PROCESSES (a) SNN (ω ) −2ω0 −ω0 (b) 0 ω0 2ω0 SNN (ω−ω0 ) −2ω0 −ω0 (c) 0 ω0 2ω0 SNN (ω+ω0 ) −2ω0 −ω0 0 ω0 2ω0 SXX (ω)=SYY (ω) (d) 0 (e) −jSXY (ω)=jSYX (ω) 0 ω Figure 5.4 Example of a bandpass process and construction of the related quadrature processes From a careful look at this example it becomes clear that care should be taken when using properties 8 and 9. This is a consequence of the operation LpfÁg, which is not always unambiguous. Although there is a different mathematical approach to avoid this, we will not go into the details here. It will be clear that no problems will arise in the case of narrowband bandpass processes. In many cases a bandpass process results from bandpass filtering of white noise. Then the spectrum of the bandpass noise is determined by the equation (see Theorem 7) SNN ð!Þ ¼ SIIð!0ÞjHð!Þj2 ð5:44Þ where SIIð!0Þ is the spectral density of the input noise and Hð!Þ the transfer function of the bandpass filter. PROBABILITY DENSITY FUNCTIONS OF THE ENVELOPE 111 It should be stressed here that the quadrature processes XðtÞ and YðtÞ are not uniquely determined; namely it follows from Equation (5.20) and Figure 5.4 that these processes are, among others, determined by the choice of the characteristic frequency !0. Finally, we will derive the relation between the spectrum of the complex envelope and the spectra of the quadrature components. The complex envelope of a stochastic bandpass process is a complex stochastic process defined by ZðtÞ ¼4 XðtÞ þ jYðtÞ ð5:45Þ Using Equations (5.26), (5.27) and (2.87) we find that RZZð Þ ¼ 2½RXXðÞ þ jRXY ð ފ ð5:46Þ and consequently SZZð!Þ ¼ 2½SXXð!Þ þ jSXY ð!ފ ð5:47Þ If SNNð!Þ is symmetrical about !0, then SXY ð!Þ ¼ 0 and the spectrum of the complex envelope reads SZZ ð!Þ ¼ 2SXXð!Þ ð5:48Þ It has been observed that the complex envelope is of importance when establishing the envelope of a bandpass signal or bandpass noise. Equation (5.48) is needed when analysing the envelope detection of (amplitude) modulated signals disturbed by noise. 5.3 PROBABILITY DENSITY FUNCTIONS OF THE ENVELOPE AND PHASE OF BANDPASS NOISE As mentioned in Section 5.2, in practice we often meet a situation that can be modelled by white Gaussian noise that is bandpass filtered. Linear filtering of Gaussian noise in turn produces Gaussian distributed noise at the filter output, and of course this holds for the special case of bandpass filtered Gaussian noise. Moreover, we conclude that the quadrature components XðtÞ and YðtÞ of Gaussian bandpass noise have Gaussian distributions as well. This is reasoned as follows. Consider the description of bandpass noise in accordance with Equation (5.20). For a certain fixed value of t, let us say t1, the random variable Nðt1Þ is constituted from a linear combination of the two random variables Xðt1Þ and Yðt1Þ, namely Nðt1Þ ¼ Xðt1Þ cos !0t1 À Yðt1Þ sin !0t1 ð5:49Þ The result Nðt1Þ can only be a Gaussian random variable if the two constituting random variables Xðt1Þ and Yðt1Þ show Gaussian distributions as well. From Equations (5.24) and (5.25) we saw that these random variables have a mean value of zero and the same variance 2, so they are identically distributed. As Xðt1Þ and Yðt1Þ are Gaussian and orthogonal 112 BANDPASS PROCESSES (see Equation (5.28)), they are independent. In Section 5.2 it was concluded that in case the bandpass filter, and thus the filtered spectrum, is symmetrical about the characteristic frequency, the cross-correlation between the quadrature components is zero and consequently the quadrature components are independent. In a number of applications the problem arises about the probability density functions of the envelope and phase of a bandpass filtered white Gaussian noise process. In ASK or FSK systems, for instance, in addition to this noise there is still a sine or cosine wave of frequency within the passband of the filter. When ASK or FSK signals are detected incoherently (which is preferred for the sake of simplicity) and we want to calculate the performance of these systems, the probability density function of the envelope of cosine (or sine) plus noise is needed. For coherent detection we need to have knowledge about the phase as well. We are therefore looking for the probability density functions of the envelope and phase of the process NðtÞ þ C cos !0t ¼ ½XðtÞ þ CŠ cos !0t À YðtÞ sin !0t ð5:50Þ where C cos !0t is the information signal and quadrature components YðtÞ and ÄðtÞ ¼4 XðtÞ þ C ð5:51Þ These quadrature components describe the process in rectangular coordinates, while we need a description on the basis of polar coordinates. When the processes for amplitude and phase are denoted by AðtÞ and ÈðtÞ, respectively, it follows that pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi AðtÞ ¼ Ä2ðtÞ þ Y2ðtÞ  ÈðtÞ ¼ arctan Y ðtÞ ÄðtÞ ð5:52Þ ð5:53Þ The conversion from rectangular to polar coordinates is depicted in Figure 5.5. The probability that the outcome of a realization ð; yÞ of the random variables ðÄ; YÞ lies in the region ða; a þ da; ;  þ dÞ is found by the coordinates transformation  ¼ a cos ; y ¼ a sin ; d dy ¼ a da d ð5:54Þ y da adφ a φ ξ Figure 5.5 Conversion from rectangular to polar coordinates PROBABILITY DENSITY FUNCTIONS OF THE ENVELOPE 113 and is written as " # fXY ðx; yÞ dx dy ¼ 1 2p2 exp À ð À CÞ2 22 þ y2 dx dy " # ¼ 1 2p2 exp À ða cos  À CÞ2 22 þ a2 sin2  a da d ð5:55Þ From this the joint probability density function follows as " # fAÈða; Þ ¼ 1 2p2 a exp À ða cos  À CÞ2 22 þ a2 sin2  ð5:56Þ The marginal probability density function of A is found by integration of this function with respect to : fAðaÞ ¼ 1 2p2 Z 2p 0 " exp À ða cos  À CÞ2 þ 22 a2 sin2 #  a da d ¼ 1 2p2 a  exp À a2 þ C2 22 Z 0 2p  Ca cos exp 2   d; a!0 ð5:57Þ In this equation the integral cannot be expressed in a closed form but is closely related to the modified Bessel function of the first kind and zero order. This Bessel function can be defined by I0ðxÞ ¼4 1 2p Z 2p 0 expðx cos Þ d ð5:58Þ Using this definition the probability density function of A is written as fAðaÞ ¼ a 2  exp À a2 þ C2 22  I0  Ca 2 ; a!0 ð5:59Þ This expression presents the probability density function for the general case of bandpass filtered white Gaussian noise added to an harmonic signal C cos !0t that lies in the passband. This distribution is called the Rice distribution. In Figure 5.6 a few Rice probability density functions are given for several values of C and for all curves where  ¼ 1. A special case can be distinguished, namely when C ¼ 0, where the signal only consists of bandpass noise since the amplitude of the harmonic signal is set to zero. Then the probability density function is fAðaÞ ¼ a 2  exp À a2 22  ; a!0 ð5:60Þ This latter probability density function corresponds to the so-called Rayleigh-distribution. Its graph is presented in Figure 5.6 as well. 114 BANDPASS PROCESSES 0.7 fA(a) 0.6 C =0 0.5 1 0.4 2 4 6 8 0.3 0.2 0.1 0 0 1 2 3 4 5 6 7 8 9 10 a Figure 5.6 Rayleigh distribution ðC ¼ 0Þ and Rice distribution for several values of C 6¼ 0 and for ¼1 Next we will calculate the probability density function of the phase. For that purpose we integrate the joint probability density function of Equation (5.56) with respect to a. The marginal probability density function of È is found as fÈðÞ ¼ 1 2p2 Z1 a¼0 a  exp À a2 À 2Ca cos  þ C2 cos2 22  þ C2 sin2   da ¼ 1 2p2 Z1 a¼0 a " exp À ða À C cos Þ2 þ 22 C2 sin2 #  da ¼ 1 2p2  exp À C2 sin2 22  Z1 a¼0 a " exp À ða À C cos 22 # Þ2 da ð5:61Þ By means of the change of the integration variable u ¼4 a À C cos   ð5:62Þ we proceed to obtain fÈðÞ ¼ ¼ ¼ 22211ppp1ee2xxppexpÀÀC2ÀC22C22si2n22sþ2in22p"1ffi2ZffiffipffiÀZffi 1CÀCc1CocscoosesðxupeþxÀpCu22ÀcoCds u2222sÞien2x2pþCÀc1uo22ÀsQZdÀu1CCcocsoesxpÀ; u2 # du 2 jj < p ð5:63Þ 2 fΦ(φ) MEASUREMENT OF SPECTRA 115 C/σ=5 1 C/σ=3 0 -π C/σ=1 C/σ=0 0 π φ Figure 5.7 Probability density function for the phase of a harmonic signal plus bandpass noise where the Q function is the well-known Gaussian integral function as defined in Appendix F. The phase probability density function is depicted in Figure 5.7. This unattractive phase equation reduces drastically when C ¼ 0 is inserted: fÈðÞ ¼ 1 2p ; jj < p ð5:64Þ Thus, it is concluded that in the case of the Rayleigh distribution the phase has a uniform probability density function. Moreover, in this case the joint probability density function of Equation (5.56) becomes independent of , so that from a formal point of view this function can be factored according to fAÈða; Þ ¼ fAðaÞ fÈðÞ. Thus, for the Rayleigh distribution, the envelope and phase are independent. This is certainly not true for the Rice distribution (C 6¼ 0). Then the expression of Equation (5.63) can, however, be simplified if the power of the sinusoidal signal is much larger than the noise variance, i.e. C ) : fÈðÞ % Cpcffiffioffiffiffis  2p   exp À C2 sin2 22 ; jj < p 2 ð5:65Þ which is approximated by a Gaussian distribution for small values of , and E½ÈŠ % 0 and E½È2Š % 2. 5.4 MEASUREMENT OF SPECTRA 5.4.1 The Spectrum Analyser The oscilloscope is the most well-known measuring instrument used to show and record how a signal behaves as a function of time. However, when dealing with stochastic signals this 116 BANDPASS PROCESSES x(t) LP filter IF power filter meter voltage controlled oscillator ramp generator display Figure 5.8 Basic scheme of the superheterodyne spectrum analyser will provide little information due to random fluctuations with time. Measurements in the frequency domain, especially the power spectrum as described in this section, is much more meaningful in these situations. Here we discuss and analyse the mode of operation of the spectrum analyser. The simplest way to measure a power spectrum is to apply the stochastic signal to a narrowband bandpass filter, to square the filter output and to record the result averaged over a sufficiently long period of time. By tuning the central frequency of the bandpass filter and recording the squared output as a function of frequency an approximation of the spectrum is achieved. However, it is very difficult to tune the frequency of such a filter over a broad range. Even more difficult is to guarantee the properties (such as a constant bandwidth) over a broad frequency range. Therefore, in modern spectrum analysers a different technique based on superheterodyning is used, which is discussed in the sequel. In Figure 5.8 the basic scheme of this spectrum analyser is presented. First the signal is applied to a lowpass filter (LP), after which it is multiplied by the signal of a local oscillator, which is a voltage controlled oscillator. The angular frequency of this oscillator is denoted by !L. Due to this multiplication the spectrum of the input signal is shifted by the value of the frequency; this shifting process has been explained in Section 3.4. Next, the shifted spectrum is filtered by a narrowband bandpass filter (IF), of which the central frequency is !IF. By keeping the filter parameters (central frequency and bandwidth) fixed but tuning the local oscillator frequency for different local oscillator frequencies, different portions of the power spectrum are filtered. This has the same effect as tuning the filter, which was suggested in the previous paragraph. The advantage of this so-called superheterodyne technique is that the way the filter filters out small portions of the spectrum is the same for all frequencies. The output of the filter is applied to a power meter. The tuning of the voltage controlled oscillator is determined by a generator that generates a linear increasing voltage as a function of time (ramp function), while the frequency is proportional to this voltage. This ramp function is used as the horizontal coordinate in the display unit, whereas the vertical coordinate is proportional to the output of the power meter. The shift of the spectrum caused by the multiplication is depicted in Figure 5.9. Looking at this figure and using Equation (3.38), we can conclude that the power after the IF filter and at a local oscillator frequency of !L is written as PY ð!LÞ ¼ ¼ 221p21pZÀA41120 ZA40201½SSXXXXðð!!ÀÀ!!LLÞÞþjHSðX!XÞðj!2 þ !Lފ d! jHð!Þj2 d! ð5:66Þ SXX (ω) 0 MEASUREMENT OF SPECTRA 117 (a) H (ω) SXX (ω−ωL) ω (b) 0 ω IF ωL Figure 5.9 (a) Spectrum to be measured; (b) spectrum after mixing and transfer function of the IF filter where Hð!Þ is the transfer function of the IF filter and A0 is the amplitude of the local oscillator signal. This shows a second advantage of the superheterodyne method; namely the recorded signal is boosted by the amplitude of the local oscillator signal, making it less vulnerable to noise during processing. If the spectrum of the input signal is supposed to be constant over the passband of the IF filter, then according to Equation (4.52) the measured power reads PY ð!LÞ % A20 4p SXX ð! IF À !LÞ jHð! IFÞj2 WN ð5:67Þ Thus, from the power indication of the instrument it appears that the spectrum at the frequency !IF À !L is approximated by SXX ð! IF À ! LÞ % A20WN 4p jHð! IFÞj2 PY ð! LÞ ð5:68Þ Here ! IF is a fixed frequency that is determined by the design of the spectrum analyser and !L is determined by the momentary tuning of the local oscillator in the instrument. Both values are known to the instrument and thus the frequency !IF À !L at which the spectral density is measured is known. Furthermore, the fraction in the right-hand member of this equation is determined by calibration and inserted in the instrument, so that reliable data can be measured. In order to gain an insight into the measured spectra a few design parameters have to be considered in more detail. The local oscillator frequency is chosen such that its value is larger than the IF frequency of the filter. From Equation (5.68) it becomes evident that in that case the frequency at which the spectrum is measured is actually negative. This is no problem at all, as the spectrum is an even function of !; in other words, that frequency may be replaced by its equal magnitude positive counterpart !L À !IF. Furthermore, the multiplication will in practice be realized by means of a so-called ‘mixer’, a non-linear device 118 BANDPASS PROCESSES (e.g. a diode). This device reproduces, besides the sum and difference frequencies, the original frequencies. In order to prevent these components from entering the bandpass filter, the IF frequency has to be higher than the highest component of the signal. The scheme shows a lowpass filter at the input of the analyser; this filter limits the frequency range of the signal that is applied to the mixer in order to make sure that the mixed signal comprises no direct components within the passband range of the IF filter. Moreover, the mixer also produces a signal of the difference frequency, by which the spectrum in Figure 5.9 shifts to the left as well. The input filter also has to prevent these components from passing into the IF filter. In modern spectrum analysers the signal to be measured is digitized and applied to a digital built-in signal processor that generates the ramp signal (see Figure 5.8). The output of the processor controls the display. This processor is also able to perform all kinds of arithmetical and mathematical operations on the measurement results, such as calculating the maximum value, averaging over many recordings, performing logarithmic scale conversion for decibel presentation, bandwidth indication, level indication, etc. 5.4.2 Measurement of the Quadrature Components The measurement of the quadrature components of a stochastic process is done by synchronous demodulation, as described in Section 3.4. The bandpass process NðtÞ is multiplied by a cosine wave as well as by a sine wave, as shown in Figure 5.10. Multiplying by a cosine as is done in the upper arm gives 2NðtÞ cosð!0tÞ ¼ 2XðtÞ cos2ð!0tÞ À 2YðtÞ sinð!0tÞ cosð!0tÞ ¼ XðtÞ þ XðtÞ cosð2!0tÞ À YðtÞ sinð2!0tÞ ð5:69Þ When the bandwidth of the quadrature components is smaller than the central frequency !0, the double frequency terms are removed by the lowpass filter, so that the component XðtÞ is left at the output. LP filter X(t ) N(t) 2cos ω0t phase shift π/2 LP filter Y(t ) Figure 5.10 Scheme for measuring the quadrature components XðtÞ and YðtÞ of a bandpass process SAMPLING OF BANDPASS PROCESSES 119 Multiplying by the sine wave in the lower arm yields À2NðtÞ sinð!0tÞ ¼ À2XðtÞ cosð!0tÞ sinð!0tÞ þ YðtÞ sin2ð!0tÞ ¼ XðtÞ sinð2!0tÞ þ YðtÞ À YðtÞ cosð2!0tÞ ð5:70Þ and removing the double frequency terms using the lowpass filter produces the YðtÞ component. If needed, the corresponding output spectra can be measured using the method described in the previous section. When the cross-power spectrum SXY ð!Þ is required we have to cross-correlate the outputs and Fourier transform the cross-correlation function. 5.5 SAMPLING OF BANDPASS PROCESSES In Section 3.5, by means of Theorem 5 it was shown that a band-limited lowpass stochastic process can be perfectly reconstructed (in the mean-squared-error sense) if the sampling rate is at least equal to the highest frequency component of the spectrum. Applying this theorem to bandpass processes would result in very high sampling rates. Intuitively, one would expect that for these processes the sampling rate should be related to bandwidth, rather than the highest frequency. This is confirmed in the next sections. Looking for a minimum sampling rate is motivated by the fact that one wants to process as little data as possible using the digital processor, owing to its limited processing and memory capacity. Once the data are digitized, filtering is performed as described in the foregoing sections. 5.5.1 Conversion to Baseband The simplest way to approach sampling of bandpass processes is to convert them to baseband. This can be performed by means of the scheme given in Figure 5.10 and described in Subsection 5.4.2. Then it can be proved that the sampling rate has to be at least twice the bandwidth of the bandpass process. This is explained as follows. If the demodulation frequency !0 in Figure 5.10 is selected in the centre of the bandpass spectrum, then the spectral width of both XðtÞ and YðtÞ is W=2, where W is the spectral width of the bandpass process. Each of the quadrature components requires a sampling rate of at least W=ð2Þ (twice its highest frequency component). Therefore, in total W= samples per second are required to determine the bandpass process NðtÞ; this is the same amount as for a lowpass process with the same bandwidth W as the bandpass process. The original bandpass process can be reconstructed from its quadrature components XðtÞ and YðtÞ by remodulating them by respectively a cosine and a sine wave and subtracting the modulated signals according to Equation (5.20). 5.5.2 Direct Sampling At first glance we would conclude that direct sampling of the bandpass process leads to unnecessarily high sampling rates. However, in this subsection we will show that direct 120 BANDPASS PROCESSES ωs ωs k k−1 1 .......... 1 k−1 k −ωH −ω0 −ωL 0 W ωL ω0 ωH ω W Figure 5.11 Direct sampling of the passband process sampling is possible at relatively low rates. This is understood when looking at Figure 5.11. The solid lines represent the spectrum of the passband process. When applying ideal sampling, this spectrum is reproduced infinitely many times, where the distance in frequency between adjacent copies is !s ¼ 2p=Ts. The spectrum of the sampled passband process can therefore be denoted as SSSð!Þ ¼ kX ¼1 k¼À1  SNN ! À k  2p Ts ð5:71Þ The spectrum SNNð!Þ comprises two portions, one centred around !0 and another centred around À!0. In Figure 5.11 the shifted versions of SNNð!Þ are presented in dotted and dashed lines. The dotted versions originate from the portion around !0 and the dashed versions from the portion around À!0. From the figure it follows that the shifted versions remain undistorted as long as they do not overlap. By inspection it is concluded that this is the case if ðk À 1Þ!s 2!L and k!s ! 2!H, where !L and !H are respectively the lower and higher frequency bounds of SNNð!Þ. Combining the two conditions yields 2!H k !s 2!L kÀ1 ð5:72Þ In order to find a minimum amount of data to be processed, one should look for the lowest value of !s that satisfies these conditions. The lowest value of !s corresponds to the largest possible value of k. In doing so one has to realize that 2 k !H and W W !L ð5:73Þ The lower bound on k is induced by the upper bound of !s and the upper bound is set to prevent the sampling rate becoming higher than would follow from the baseband sampling theorem. For the same reason the condition on W was introduced in Equations (5.73). Example 5.2: Suppose that we want to sample a bandpass process of bandwidth 50 MHz and with a central frequency of 1 GHz. This means that !L=ð2pÞ ¼ 975 MHz and !H=ð2pÞ ¼ 1025 MHz. From PROBLEMS 121 Equations (5.73) it can easily be seen that the maximum allowable value is k ¼ 20. The limits of the sample frequency are 102:5MHz < !s=ð2pÞ < 102:63 MHz, which follows from Equation (5.72). The possible minimum values of the sample frequency are quite near to what is found when applying the conversion to the baseband method from Section 5.5.1, which results in a sampling rate of 100 MHz. & 5.6 SUMMARY The chapter starts with a summary of the description of deterministic bandpass signals, where the so-called quadrature components are introduced. Bandpass signals or processes mostly result from the modulation of signals. Several modulation methods are briefly and conceptually described. Different complex description methods are introduced, such as the analytical signal and the complex envelope. Moreover, the baseband equivalent transfer function and impulse response of passband systems are defined. When applying all these concepts, their relation to the physical meaning should always be kept in mind when interpreting results. For bandpass processes all these concepts are further exploited. Both in time and frequency domains a number of properties of the quadrature components are presented, together with their relation to the bandpass process. The probability density function of the amplitude and phase are derived for bandpass filtered white Gaussian noise. These are of importance when dealing with modulated signals that are disturbed by noise. The measurement of spectra is done using a so-called spectrum analyser. Since this equipment uses heterodyning and subsequently passband filtering, we use what we learned earlier in this chapter to describe its operation. The instrument is a powerful tool in the laboratory. Finally, we describe the sampling of passband processes; two methods are given. It is proved that the minimum sampling rate of these processes equals twice the bandwidth. 5.7 PROBLEMS 5.1 It is well known that baseband systems provide distortionless transmission if the amplitude of the transfer characteristic is constant and the phase shift a linear function of frequency over the frequency band of the information signal [6, 13]. Let us now consider the conditions for distortionless transmission via a bandpass system. We call the transmission distortionless when on transmission the signal shape is preserved, but a delay in it may be allowed. It will be clear that once more the amplitude characteristic of the system has to be constant over the signal band. Furthermore, assume that the phase characteristic of the transfer function over the information band can be written as ð!Þ ¼ Àj0!0 À jgð! À !0Þ 122 BANDPASS PROCESSES R L C Figure 5.12 with 0 and g constants. (a) Show that, besides the amplitude condition, this phase condition provides distortionless transmission of signals modulated by the carrier frequency !0. (b) What consequences have 0 and g for the output signal? 5.2 The circuit shown in Figure 5.12 is a bandpass filter. (a) Derive the transfer function Hð!Þ. pffiffiffiffiffiffiffiffi (b) Derive jHð!Þj and use Matlab to plot it for LC ¼ 0:01 and R C=L ¼ 10; 20; 50. (c) Let us call the resonance frequency !0. Determine the equivpalffieffiffinffiffitffiffiffibaseband transfer function and plot both its amplitude and phase for R C=L ¼ 10; 50, while preserving the value of LC. (d) Determine for the values from (c) the phase delay (0 from Problem 5.1) and plot the group delay (g from Problem 5.1). 5.3 A bandpass process NðtÞ has the following power spectrum: 8 >< P cos½p ð! À !0Þ=WŠ; ÀW=2 ! À !0 W=2 SNN ð!Þ ¼ >: P cos½p 0; ð! þ !0Þ=W Š; ÀW=2 ! þ !0 W=2 all other values of ! where P, W and !0 > W are positive, real constants. (a) What is the power of NðtÞ? (b) What is the power spectrum SXXð!Þ of XðtÞ if NðtÞ is represented as in Equation (5.20)? (c) Calculate the cross-correlation function RXY ðÞ. (d) Are the quadrature processes XðtÞ and YðtÞ orthogonal? 5.4 White noise with spectral density of N0=2 is applied to an ideal bandpass filter with the central passband radial frequency !0 and bandwidth W. (a) Calculate the autocorrelation function of the output process. Use Matlab to plot it. (b) This output is sampled and the sampling instants are given as tn ¼ n  2p=W with n integer values. What can be said about the sample values? 5.5 A bandpass process is represented as in Equation (5.20) and has the power spectrum according to Figure 5.13; assume that !1 > W. (a) Sketch SXXð!Þ and SXY ð!Þ when !0 ¼ !1. (b) Repeat (a) when !0 ¼ !2. −ω2 −ω1 PROBLEMS 123 SNN (ω) S0 ω ω2 0 ω1−W /2 ω1 ω1 + W/2 Figure 5.13 SXX (ω) S0 −W 0 W ω H (ω) H2 H1 0 ω1 ω2 ω3 ω Figure 5.14 5.6 A wide-sense stationary process XðtÞ has a power spectrum as depicted in the upper part of Figure 5.14. This process is applied to a filter with the transfer function Hð!Þ as given in the lower part of the figure. The data for the spectrum and filter are: S0 ¼ 10À6, W ¼ 2p  107, !1 ¼ 2p  0:4  107, !2 ¼ 2p  0:5  107, !3 ¼ 2p  0:6  107, H1 ¼ 2 and H2 ¼ 3. (a) Determine the power spectrum of the output. (b) Sketch the spectra of the quadrature components of the output when !0 ¼ !1. (c) Calculate the power of the output process. 5.7 A wide-sense stationary white Gaussian process has a spectral density of N0=2. This process is applied to the input of the linear time-invariant filter. The filter has a bandpass characteristic with the transfer function Hð!Þ ¼  1; 0; !0 À W=2 < j!j < !0 þ W=2 elsewhere where !0 > W. (a) Sketch the transfer function Hð!Þ. (b) Calculate the mean value of the output process. (c) Calculate the variance of the output process. 124 BANDPASS PROCESSES SN1N1(ω) A −W 0 W ω Figure 5.15 (d) Determine and dimension the probability density function of the output. (e) Determine the power spectrum and the autocorrelation function of the output. 5.8 The wide-sense stationary bandpass noise process N1ðtÞ has the central frequency !0. It is modulated by an harmonic carrier to form the process N2ðtÞ ¼ N1ðtÞ cosð!0t À ÂÞ where  is independent of N1ðtÞ and is uniformly distributed on the interval ð0; 2pŠ. (a) Show that N2ðtÞ comprises both a baseband component and a bandpass compo- nent. (b) Calculate the mean values and variances of these components, expressed in terms of the properties of N1ðtÞ. 5.9 The noise process N1ðtÞ is wide-sense stationary. Its spectral density is given in Figure 5.15. By means of this process a new process N2ðtÞ is produced according to N2ðtÞ ¼ N1ðtÞ cosð!0t À ÂÞ À N1ðtÞ sinð!0t À ÂÞ where  is a random variable that is uniformly distributed on the interval ð0; 2pŠ. Calculate and sketch the spectral density of N2ðtÞ. 5.10 Consider the stochastic process NðtÞ ¼ XðtÞ cosð!0t À ÂÞ À YðtÞ sinð!0t À ÈÞ with !0 a constant. The random variables  and È are independent of XðtÞ and YðtÞ and uniformly distributed on the interval ð0; 2pŠ. The spectra SXXð!Þ, SYY ð!Þ and SXY ð!Þ are given in Figure 5.16, where WY < WX < !0 and in the right-hand picture the solid line is the real part of SXY ð!Þ and the dashed line is its imaginary part. (a) Determine and sketch the spectrum SNNð!Þ in the case where  and È are independent. (b) Determine and sketch the spectrum SNNð!Þ in the case where  ¼ È. SXX (ω) 1 SYY (ω) 1 PROBLEMS 125 SXY(ω) 1 −WX 0 WX ω −WY 0 WY ω −WY 0 WY ω Figure 5.16 SNN(ω) 1 −7 −5 −4 0 45 7 ω Figure 5.17 5.11 Consider the wide-sense stationary bandpass process NðtÞ ¼ XðtÞ cosð!0tÞ À YðtÞ sinð!0tÞ where XðtÞ and YðtÞ are baseband processes. The spectra of these processes are  1; j!j < W SXXð!Þ ¼ SYY ð!Þ ¼ 0; j!j ! W and SXY ð!Þ ¼  j ! W 0; ; j!j < W j!j ! W where W < !0. (a) Sketch the spectra SXXð!Þ, SYY ð!Þ and SXY ð!Þ. (b) Show how SNN ð!Þ can be reconstructed from SXXð!Þ and SXY ð!Þ. Sketch SNN ð!Þ. (c) Sketch the spectrum of the complex envelope of NðtÞ. (d) Calculate the r.m.s. bandwidth of the complex envelope ZðtÞ. 5.12 A wide-sense stationary bandpass process has the spectrum as given in Figure 5.17. The characteristic frequency is !0 ¼ 5 rad/s. (a) Sketch the power spectra of the quadrature processes. (b) Are the quadrature processes uncorrelated? (c) Are the quadrature processes independent? 126 BANDPASS PROCESSES 5.13 A wide-sense stationary bandpass process is given by NðtÞ ¼ XðtÞ cosð!0tÞ À YðtÞ sinð!0tÞ where XðtÞ and YðtÞ are independent random signals with an equal power of Ps and bandwidth W < !0. These signals are received by a synchronous demodulator scheme as given in Figure 5.10; the lowpass filters are ideal filters, also of bandwidth W. The received signal is disturbed by additive white noise with spectral density N0=2. Calculate the signal-to-noise ratios at the outputs. 5.14 The power spectrum of a narrowband wide-sense stationary bandpass process NðtÞ needs to be measured. However, there is no spectrum analyser available that covers the frequency range of this process. Two product modulators are available, based on which the circuit of Figure 5.10 is constructed and the oscillator is tuned to the central frequency of NðtÞ. The LP filters allow frequencies smaller than W to pass unattenuated and block higher frequencies completely. By means of this set-up and a lowfrequency spectrum analyser the spectra shown in Figure 5.18 are measured. Reconstruct the spectrum of NðtÞ. 5.15 The spectrum of a bandpass signal extends from 15 to 25 MHz. The signal is sampled with direct sampling. (a) What is the range of possible sampling frequencies? (b) How much higher is the minimum direct sampling frequency compared with the minimum frequency when conversion to baseband is applied. (c) Compare the former two sampling frequencies with that following from the Nyquist baseband sampling theorem (Theorems 4 and 5). 5.16 A baseband signal of bandwidth 1 kHz is modulated on a carrier frequency of 8 kHz. (a) Sketch the spectrum of the modulated bandpass signal. (b) What is the minimum sampling frequency based on the Nyquist baseband sampling theorem (Theorems 4 and 5). (c) What is the minimum sampling frequency based on direct sampling. SXX (ω) 1 −jSXY (ω) 1 −W 0 Wω −W 0 W ω −1 Figure 5.18 5.17 The transfer function of a discrete-time filter is given by PROBLEMS 127 H~ ðzÞ ¼ 1 À 1 0:2zÀ1 þ 0:95zÀ2 (a) Is this a stable system? (b) Use Matlab’s freqz to plot the absolute value of the transfer function. (c) What type of filter is this? Search for the maximum value of the transfer function. (d) Suppose that to the input of this filter a sinusoidal signal is applied with a frequency where the absolute value of the transfer function has its maximum value. Moreover, suppose that this signal is disturbed by wide-sense stationary white noise such that the signal-to-noise ratio amounts to 0 dB. Calculate the signal-to-noise ratio at the filter output. (e) Explain the difference in signal-to-noise ratio improvement compared to that of Problem 4.28. 6 Noise in Networks and Systems Many electrical circuits generate some kind of noise internally. The most well-known kind of noise is thermal noise produced by resistors. Besides this, several other kinds of noise sources can be identified, such as shot noise and partition noise in semiconductors. In this chapter we will describe the thermal noise generated by resistors, while shot noise is dealt with in Chapter 8. We shall show how internal noise sources can be transferred to the output terminals of a network, where the noise becomes observable to the outside world. For that purpose we shall consider the cascading of noisy circuits as well. In many practical situations, which we refer to in this chapter, a noise source can adequately be described on the basis of its power spectral density; this spectrum can be the result of a calculation or the result of a measurement as described in Section 5.4. 6.1 WHITE AND COLOURED NOISE Realization of a wide-sense stationary noise process NðtÞ is called white noise when the power spectral density of NðtÞ has a constant value for all frequencies. Thus, it is a process for which SNN ð!Þ ¼ N0 2 ð6:1Þ holds, with N0 a real positive constant. By applying the inverse Fourier transform to this spectrum, the autocorrelation function of such a process is found to be RNN ð Þ ¼ N0 2 ð Þ ð6:2Þ The name white noise was taken from optics, where white light comprises all frequencies (or equivalently all wavelengths) in the visible region. Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 130 NOISE IN NETWORKS AND SYSTEMS It is obvious that white noise cannot be a meaningful model for a noise source from a physical point of view. Looking at Equation (3.8) reveals that such a process would comprise an infinitely large amount of power, which is physically impossible. Despite the shortcomings of this model it is nevertheless often used in practice. The reason is that a number of important noise sources (see, for example, Section 6.2) have a flat spectrum over a very broad frequency range. Deviation from the white noise model is only observed at very high frequencies, which are of no practical importance. The name coloured noise is used in situations where the power spectrum is not white. Examples of coloured noise spectra are lowpass, highpass and bandpass processes. 6.2 THERMAL NOISE IN RESISTORS An important example of white noise is thermal noise. This noise is caused by thermal movement (or Brownian motion) of the free electrons in each electrical conductor. A resistor with resistance R at an absolute temperature of T has at its open terminals a noise voltage with a Gaussian probability density function with a mean value of zero and of which the power spectral density is SVV ð!Þ ¼ h Rhj!j p exp hj!j 2pkT À i 1 ½V2 sŠ ð6:3Þ where k ¼ 1:38  10À23 ½J=KŠ ð6:4Þ is the Boltzmann constant and h ¼ 6:63  10À34 ½J sŠ ð6:5Þ is the Planck constant. Up until frequencies of 1012 Hz the expression (6.3) has an almost constant value, which gradually decreases to zero beyond that frequency. For useful frequencies in the radio, microwave and millimetre wavelength ranges, the power spectrum is white, i.e. flat. Using the well-known series expansion of the exponential in Equation (6.3), a very simple approximation of the thermal noise in a resistor is found. Theorem 9 The spectrum of the thermal noise voltage across the open terminals of resistance R which is at the absolute temperature T is SVV ð!Þ ¼ 2kTR ½V2 sŠ ð6:6Þ This expression is much simpler than Equation (6.3) and can also be derived from physical considerations, which is beyond the scope of this text. THERMAL NOISE IN PASSIVE NETWORKS 131 6.3 THERMAL NOISE IN PASSIVE NETWORKS In Chapter 4 the response of a linear system to a stochastic process has been analysed. There it was assumed that the system itself was noise free, i.e. it does not produce noise itself. In the preceding section, however, we indicated that resistors produce noise; the same holds for semiconductor components such as transistors. Thus, if these components form part of a system, they will contribute to the noise at the output terminals. In this section we will analyse this problem. In doing so we will confine ourselves to the influence of thermal noise in passive networks. In a later section active circuits will be introduced. As a model for a noisy resistor we introduce the equivalent circuit model represented in Figure 6.1. This model shows a noise-free resistance R in series with a noise process VðtÞ, for which Equation (6.6) describes the power spectral density. This scheme is called The´venin’s equivalent voltage model. From network theory we know that a resistor in series with a voltage source can also be represented as a resistance R in parallel with a current source. The magnitude of this current source is IðtÞ ¼ VðtÞ R ð6:7Þ In this way we arrive at the scheme given in Figure 6.2. This model is called Norton’s equivalent current model. Using Equations (4.27) and (6.7) the spectrum of the current source is obtained. Theorem 10 The spectrum of the thermal noise current when short-circuiting a resistance R that is at the absolute temperature T is SII ð!Þ ¼ 2kT R ½A2 sŠ ð6:8Þ In both schemes of Figures 6.2 and 6.1, the resistors are assumed to be noise free. When calculating the noise power spectral density at the output terminals of a network, the following method is used. Replace all noisy resistors by noise-free resistors in series with SVV (ω) = 2kTR [V2s] V(t) R Figure 6.1 The´venin equivalent voltage circuit model of a noisy resistor 132 NOISE IN NETWORKS AND SYSTEMS SII (ω) = 2kT/R [A2s] I(t) R Figure 6.2 Norton equivalent current circuit model of a noisy resistor a voltage source (according to Figure 6.1) or parallel with a current source (according to Figure 6.2). The schemes are equivalent, so it is possible to select the more convenient of the two schemes. Next, the transfer function from the voltage source or current source to the output terminals is calculated using network analysis methods. Invoking Equation (4.27), the noise power spectral density at the output terminals is found. Example 6.1: Consider the circuit presented in Figure 6.3. We wish to calculate the mean squared value of the voltage across the capacitor. Express Vcð!Þ in terms of V using the relationship Vcð!Þ ¼ Hð!Þ Vð!Þ ð6:9Þ and Hð!Þ ¼ Vcð!Þ V ð!Þ ¼ 1 j!C 1 j!C þ R ¼ 1 þ 1 j!RC ð6:10Þ R C V(t ) C R Vc (t ) (a) (b) Figure 6.3 (a) Circuit to be analysed; (b) The´venin equivalent model of the circuit THERMAL NOISE IN PASSIVE NETWORKS 133 Invoking Equation (4.27), the power spectral density of Vcð!Þ reads SVc Vc ð!Þ ¼ 2kTR 1 þ 1 !2R2C2 ½V2 sŠ and using Equation (4.28) PVc ¼ 1 2p Z1 À1 1 2kTR þ !2R2C2 d! ¼ kT C ½V2 sŠ ð6:11Þ ð6:12Þ & When the network comprises several resistors, then these resistors will produce their noise independently from each other; namely the thermal noise is a consequence of the Brownian motion of the free electrons in the resistor material. As a rule the Brownian motion of electrons in one of the resistors will not be influenced by the Brownian motion of the electrons in different resistors. Therefore, at the output terminals the different spectra resulting from the several resistors in the circuit may be added. Let us now consider the situation where a resistor is loaded by a second resistor (see Figure 6.4). If the loading resistance is called RL, then similar to the method presented in Example 6.1, the power spectral density of the voltage V across RL due to the thermal noise produced by R can be calculated. This spectral density is found by applying Equation (4.27) to the circuit of Figure 6.4, i.e. inserting the transfer function from the noise source to the load resistance SVV ð!Þ ¼ 2kTR jHð!Þj2 ¼ 2kTR ðR R2L þ RLÞ2 ½V2 sŠ ð6:13Þ Note the confusion that may arise here. When talking about the power of a stochastic process in terms of stochastic process theory, the expectation of the quadratic of the stochastic process is implied. This nomenclature is in accordance with Equation (6.13). However, when speaking about the physical concept of power, then conversion from the stochastic theoretical concept of power is required; this conversion will in general be simply multiplication by a constant factor. As for electrical power dissipated in a resistance RL, we thermal noise source R RL V Figure 6.4 A resistance R producing thermal noise and loaded by a resistance RL 134 NOISE IN NETWORKS AND SYSTEMS have the formulas P ¼ V2=RL ¼ I2RL, and the conversion reads as SPð!Þ ¼ 1 RL SVV ð!Þ ¼ RL SII ð!Þ ½W sŠ ð6:14Þ For the spectral density of the electrical power that is dissipated in the resistor RL we have SPð!Þ ¼ 2kTR ðR RL þ RLÞ2 ½W sŠ ð6:15Þ It is easily verified that the spectral density given by Equation (6.15) achieves its maximum when R ¼ RL and the density is SPmax ð!Þ ¼4 Sað!Þ ¼ kT 2 ½W sŠ ð6:16Þ Therefore, the maximum power spectral density from a noisy resistor transferred to an external load amounts to kT=2, and this value is called the available spectral density. It can be seen that this spectral density is independent of the resistance value and only depends on temperature. Analogously to Equation (6.6), white noise sources are in general characterized as SVV ð!Þ ¼ 2kTeRe ½V2 sŠ ð6:17Þ In this representation the noise spectral density may have a larger value than the one given by Equation (6.6), due to the presence of still other noise sources than those caused by that particular resistor. We consider two different descriptions: 1. The spectral density is related to the value of the physical resistance R and we define Re ¼ R. In this case Te is called the equivalent noise temperature; the equivalent noise temperature may differ from the physical temperature T. 2. The spectral density is related to the physical temperature T and we define Te ¼ T. In this case Re is called the equivalent noise resistance; the equivalent noise resistance may differ from the physical value R of the resistance. In networks comprising reactive components such as capacitors and coils, both the equivalent noise temperature and the equivalent noise resistance will generally depend on frequency. An example of this latter situation is elucidated when considering a generalization of Equation (6.6). For that purpose consider a circuit that only comprises passive components (R, L, C and an ideal transformer). The network may comprise several of each of these items, but it is assumed that all resistors are at the same temperature T. A pair of terminals constitute the output of the network and the question is: what is the power spectral density of the noise at the output of the circuit as a consequence of the thermal noise generated by the different resistors (hidden) in the circuit? The network is considered as a multiport; when the network comprises n resistors then we consider a multiport circuit with n þ 1 terminal pairs. The output terminals are denoted by the terminal pair numbered 0. I1 V1 Ii Vi THERMAL NOISE IN PASSIVE NETWORKS 135 R1 I0 Ri MULTI-PORT V0 In Rn Vn Figure 6.5 A network comprising n resistors considered as a multiport Next, all resistors are put outside the multiport but connected to it by means of the terminal pairs numbered from 1 to n (see Figure 6.5). The relations between the voltages across the terminals and the currents flowing in or out of the multiport via the terminals are denoted using standard well-known network theoretical methods: I0 ¼ Y00V0 þ Á Á Á þ Y0iVi þ Á Á Á þ Y0nVn ÁÁÁ ¼ ÁÁÁ ÁÁÁ ÁÁÁ ÁÁÁ ÁÁÁ Ii ¼ Yi0V0 þ Á Á Á þ YiiVi þ Á Á Á þ YinVn ÁÁÁ ¼ ÁÁÁ ÁÁÁ ÁÁÁ ÁÁÁ ÁÁÁ In ¼ Yn0V0 þ Á Á Á þ YniVi þ Á Á Á þ YnnVn ð6:18Þ Then it follows for the unloaded voltage at the output terminal pair 0 that V0open ¼ Xn À Y0i i¼1 Y00 Vi ð6:19Þ The voltages, currents and admittances in Equations (6.18) and (6.19) are functions of !. They represent voltages, currents and the relations between them when the excitation is a harmonic sine wave with angular frequency !. Therefore, we may also write as an alternative to Equation (6.19) Xn V0open ¼ Hið!ÞVi i¼1 ð6:20Þ When the voltage Vi is identified as the thermal noise voltage produced by resistor Ri, then the noise voltage at the output results from the superposition of all noise voltages originating from several resistors, each of them being filtered by a different transfer function Hið!Þ. As observed before, we suppose the noise contribution from a certain resistor to be independent 136 NOISE IN NETWORKS AND SYSTEMS of these of all other resistors. Then the power spectral density of the output noise voltage is SV0V0 ¼ Xn jHið!Þj2SViVi i¼1 ¼ X i¼n1 YY000i 2SViVi ¼ 2kT X i¼n1 YY000i 2Ri ð6:21Þ It appears that the summation may be substantially reduced. To this end consider the situation where the resistors are noise free (i.e. Vi ¼ 0 for all i 6¼ 0) and where the voltage V0 is applied to the terminal pair 0. From Equation (6.18) it follows in this case that Ii ¼ Yi0V0 ð6:22Þ The dissipation in resistor Ri becomes Pi ¼ jIij2Ri ¼ jYi0j2jV0j2Ri ½WŠ ð6:23Þ As the multiport itself does not comprise any resistors, the total dissipation in the resistors has to be produced by the source that is applied to terminals 0, or Xn jI0j2RefZ0g ¼ jV0j2 jYi0j2Ri i¼1 ð6:24Þ where RefZ0g is the real part of the impedance of the multiport observed at the output terminal pair. As a consequence of the latter equation Xn i¼1 jYi0j2 jY00j2 Ri ¼ RefZ0g ð6:25Þ Passive networks are reciprocal, so that Yi0 ¼ Y0i. Substituting this into Equation (6.25) and the result from Equation (6.21) yields the following theorem. Theorem 11 If in a passive network comprising several resistors, capacitors, coils and ideal transformers all resistors are at the same temperature T, then the voltage noise spectral density at the open terminals of this network is SVV ð!Þ ¼ 2kT RefZ0g ½V2 sŠ ð6:26Þ where Z0 is the impedance of the network at the open terminal pair. This generalization of Equation (6.6) is called Nyquist’s theorem. Comparing Equation (6.26) with Equation (6.17) and if we take Te ¼ T, then the equivalent noise resistance becomes equal to RefZ0g. When defining this quantity we emphasized that it can be frequency dependent. This is further elucidated when studying Example 6.1, which is presented in Figure 6.3. SYSTEM NOISE 137 Example 6.2: Let us reconsider the problem presented in Example 6.1. The impedance at the terminals of Figure 6.3(a) reads Z0 ¼ 1 þ R j!RC ð6:27Þ with its real part RefZ0g ¼ 1 þ R !2R2C2 ð6:28Þ Substituting this expression into Equation (6.26) produces the voltage spectral density at the terminals SVc Vc ð!Þ ¼ 2kT 1 þ R !2R2C2 ½V2 sŠ ð6:29Þ As expected, this is equal to the expression of Equation (6.11). & Equation (6.26) is a description according to The´venin’s equivalent circuit model (see Figure 6.1). A description in terms of Norton’s equivalent circuit model is possible as well (see Figure 6.2). Then SI0I0 ð!Þ ¼ SV0 V0 jZ0j2 ¼ 2kT RefY0 g ½A2 sŠ ð6:30Þ where Y0 ¼4 1 Z0 ð6:31Þ Equation (6.30) presents the spectrum of the current that will flow through the shortcut that is applied to a certain terminal pair of a network. Here Y0 is the admittance of the network at the shortcut terminal pair. 6.4 SYSTEM NOISE The method presented in the preceding section can be applied to all noisy components in amplifiers and other subsystems that constitute a system. However, this leads to very extensive and complicated calculations and therefore is of limited value. Moreover, when buying a system the required details for such an analysis are not available as a rule. There is therefore a need for an alternative more generic noise description for (sub)systems in terms of relations between the input and output. Based on this, the quality of components, such as amplifiers, can be characterized in terms of their own noise contribution. In this way the noise behaviour of a system can be calculated simply and quickly. 138 NOISE IN NETWORKS AND SYSTEMS 6.4.1 Noise in Amplifiers In general, amplifiers will contribute considerably to noise in a system, owing to the presence of noisy passive and active components in it. In addition, the input signals of amplifiers will in many cases also be disturbed by noise. We will start our analysis by considering ideal, i.e. noise-free, amplifiers. The most general equivalent scheme is presented in Figure 6.6. For the sake of simplifying the equations we will assume that all impedances in the scheme are real. A generalization to include reactive components is found in reference [4]. The amplifier has an input impedance of Ri, an output impedance of Ro and a transfer function of Hð!Þ. The source has an impedance of Rs and generates as open voltage a wide-sense stationary stochastic voltage process Vs with the spectral density Sssð!Þ. This process may represent noise or an information signal, or a combination (addition) of these types of processes. The available spectral density of this source is Ssð!Þ ¼ Sssð!Þ=ð4RsÞ. This follows from Equation (6.15) where the two resistances are set at the same value Rs. Using Equation (4.27), the available spectral density at the output of the amplifier is found to be Soð!Þ ¼ Sooð!Þ 4Ro ¼ jHð!Þj2 Siið!Þ 4Ro ¼ jHð!Þj2 4Ro  Rs Ri þ 2 Ri Sssð!Þ ð6:32Þ The available power gain of the amplifier is defined as the ratio of the available spectral densities of the sources from Figure 6.6: Gað!Þ ¼4 Soð!Þ Ssð!Þ ¼ Sooð!Þ Sssð!Þ Rs Ro ¼  jHð!Þj2 Rs Ri þ 2 Rs Ri Ro ð6:33Þ In case the impedances at the input and output are matched to produce maximum power transfer (i.e. Ri ¼ Rs and Ro ¼ RL), the practically measured gain will be equal to the available gain. Now we will assume that the input source generates white noise, either from a thermal noise source or not, with an equivalent noise temperature of Ts. Then Ssð!Þ ¼ kTs=2 and the available spectral density at the output of the amplifier, supposed to be noise free, may be written as Soð!Þ ¼ Gað!ÞSsð!Þ ¼ Gað!Þ kTs 2 ð6:34Þ AMPLIFIER Rs Ro Vs Vi Ri Vo = H(ω)Vi RL Figure 6.6 Model of an ideal (noise-free) amplifier with noise input Ss(ω) = kTs/2 Ga(ω) (a) SYSTEM NOISE 139 + So(ω) Sint(ω) Ss(ω) = kTs/2 + G kTe/2 (b) filter of bandwidth WN PNo Figure 6.7 Block schematic of a noisy amplifier with the amplifier noise positioned (a) at the output or (b) at the input From now on we will assume that the amplifier itself produces noise as well and it seems to be reasonable to suppose that the amplifier noise is independent of the noise generated by the source Vs. Therefore the available output spectral density is Soð!Þ ¼ Gað!Þ kTs 2 þ Sintð!Þ ð6:35Þ where Sintð!Þ is the available spectral density at the output of the amplifier as a consequence of the noise produced by the internal noise sources present in the amplifier itself. The model that corresponds to this latter expression is drawn in Figure 6.7(a). The total available noise power at the output is found by integrating the output spectral density PNo ¼ 1Z1 2p À1 Soð!Þ d! ¼ 1 kTs 2p 2 Z1 À1 Gað!Þ d! þ 1Z1 2p À1 Sintð!Þ d! ð6:36Þ This output noise power will be expressed in terms of the equivalent noise bandwidth (see Equation (4.51)) for the sake of simplifying the notation. For !0 we substitute that value for which the gain is maximal and we denote at that value Gað!0Þ ¼ G. Then it is found that 1 p GWN ¼ 1Z1 2p À1 Gað!Þ d! ð6:37Þ Using this latter equation the first term of the right-hand side of Equation (6.36) can be written as GkTsWN=ð2pÞ. In order to be able to write the second term of that equation in a similar way the effective noise temperature of the amplifier is defined as Te ¼4 1 GkWN Z1 À1 Sintð!Þ d! ð6:38Þ Based on this latter equation the total noise power at the output is written as PNo ¼ GkTs WN 2p þ GkTe WN 2p ¼ GkðTs þ TeÞ WN 2p ð6:39Þ 140 NOISE IN NETWORKS AND SYSTEMS It is emphasized that WN=ð2pÞ represents the equivalent noise bandwidth in hertz. By the representation of Equation (6.39) the amplifier noise is in the model transferred to the input (see Figure 6.7(b)). In this way it can immediately be compared with the noise generated by the source at the input, which is represented by the first term in Equation (6.39). 6.4.2 The Noise Figure Let us now consider a noisy device, an amplifier or a passive device, and let us suppose that the device is driven by a source that is noisy as well. The noise figure F of the device is defined as the ratio of the total available output noise spectral density (due to both the source and device) and the contribution to that from the source alone, in the later case supposing that the device is noise free. In general, the two noise contributions can have frequencydependent spectral densities and thus the noise figure can also be frequency dependent. In that case it is called the spot noise figure. Another definition of the noise figure can be based on the ratio of the two total noise powers. In that case the corresponding noise figure is called the average noise figure. In many situations, however, the noise sources can be modelled as white sources. Then, based on the definition and Equation (6.39), it is found that F ¼ Ts þ Te ¼ 1 þ Te Ts Ts ð6:40Þ It will be clear that different devices can have different effective noise temperatures; this depends on the noise produced by the device. However, suppliers want to specify the quality of their devices for a standard situation. Therefore the standard noise figure for the situation where the source is at room temperature is defined as Fs ¼ 1 þ Te T0 ; with T0 ¼ 290 K ð6:41Þ Thus for a very noisy device the effective noise temperature is much higher than room temperature ðTe ) T0Þ and Fs ) 1. This does not mean that the physical temperature of the device is very high; this can and will, in general, be room temperature as well. Especially for amplifiers, the noise figure can also be expressed in terms of signal-tonoise ratios. For that purpose the available signal power of the source is denoted by Ps, so that the signal-to-noise ratio at the input reads  S N ¼ s Ps kTs WN 2p ð6:42Þ Note that the input noise power has only been integrated over the equivalent noise bandwidth WN of the amplifier, although the input noise power is actually unlimited. This procedure is followed in order to be able to compare the input and output noise power based on the same bandwidth; for the output noise power it does not make any difference. It is an obvious choice to take for this bandwidth, the equivalent noise bandwidth, as this will reflect the actual noise power at the output. Furthermore, it is assumed that the signal spectrum is limited to the same bandwidth, so that the available signal power at the output is denoted as SYSTEM NOISE 141 Pso ¼ GPs. Using Equation (6.39) we find that the signal-to-noise ratio at the output is  S N ¼ o GPs PNo ¼ GPs GkðTs þ TeÞ WN 2p ð6:43Þ This signal-to-noise ratio is related to the signal-to-noise ratio at the input as   S N ¼ o ð1 þ Ps Te Ts ÞkTs WN 2p ¼ 1 1 þ Te Ts S N s ð6:44Þ As the first factor in this expression is always smaller than 1, the signal-to-noise ratio is always deteriorated by the amplifier, which may not surprise us. This deterioration depends on the value of the effective noise temperature compared to the equivalent noise temperature of the source. If, for example, Te ( Ts, then the signal-to-noise ratio will hardly be reduced and the amplifier behaves virtually as a noise-free component. From Equation (6.44), it follows that ÀÁ S ÀNÁs S No ¼ 1 þ Te Ts ¼ F ð6:45Þ Note that the standard noise figure is defined for a situation where the source is at room temperature. This should be kept in mind when determining F by means of a measurement. Suppliers of amplifiers provide the standard noise figure as a rule in their data sheets, mostly presented in decibels (dB). Sometimes, the noise figure is defined as the ratio of the two signal-to-noise ratios given in Equation (6.45). This can be done for amplifiers but can cause problems when considering a cascade of passive devices such as attenuators, since in that case input and output are not isolated and the load impedance of the source device is also determined by the load impedance of the devices. Example 6.3: As an interesting and important example, we investigate the noise figure of a passive twoport device such as a cable or an attenuator. Since the two-port device is passive it is reasonable to suppose that the power gain is smaller than 1 and denoted as G ¼ 1=L, where L is the power loss of the two-port device. The signal-to-noise ratio at the input is written as in Equation (6.42), while the output signal power is by definition Pso ¼ Ps=L. The passive twoport device is assumed to be at temperature Ta. The available spectral density of the output noise due to the noise contribution of the two-port device itself is determined by the impedance of the output terminals, according to Theorem 11. This spectral density is kTa=2. The contribution of the source to the output available spectral density is kTs=ð2LÞ. However, since the resistance Rs of the input circuit is part of the impedance that is observed at the output terminals, the portion kTa=ð2LÞ of its noise contribution to the output is already involved in the noise kTa=2, which follows from the theorem. Only compensation for the difference in temperature is needed; i.e. we have to include an extra portion kðTs À TaÞ=ð2LÞ. 142 NOISE IN NETWORKS AND SYSTEMS Now the output signal-to-noise ratio becomes  S¼ No ½kTa þ k L Ps L ðTs À Taފ WN 2p After some simple calculations the noise figure follows from the definition ð6:46Þ Fs ¼ 1 þ ðL À 1Þ Ta Ts ; with Ts ¼ T0 When the two-port device is at room temperature this expression reduces to ð6:47Þ Fs ¼ L ð6:48Þ It is therefore concluded that the noise figure of a passive two-port device equals its power loss. & 6.4.3 Noise in Cascaded Systems In this subsection we consider the cascade connection of systems that may comprise several noisy amplifiers and other noisy components. We look for the noise properties of such a cascade connection, expressed as the parameters of the individual components as they are developed in the preceding subsection. In order to guarantee that the maximum power transfer occurs from one device to another, we assume that the impedances are matched; i.e. the input impedance of a device is the complex conjugate (see Problem 6.7) of the output impedance of the driving device. For the time being and for the sake of better understanding we only consider here a simple configuration consisting of the cascade of two systems (see Figure 6.8). The generalization to a cascade of more than two systems is quite simple, as will be shown later on. In the figure the relevant quantities of the two systems are indicated; they are the maximum power gain Gi, the effective noise temperature Tei and the equivalent noise bandwidth Wi. The subscripts i refer to system 1 for i ¼ 1 and to system 2 for i ¼ 2, while the noise first enters system 1 and the output of system 1 is connected to the input of system 2 (see Figure 6.8). We assume that the passband of system 2 is completely encompassed by that of system 1, and as a consequence W2 W1. This condition guarantees that all systems contribute to the output noise via the same bandwidth. Therefore, the equivalent noise bandwidth is equal to that of system 2: WN ¼ W2 ð6:49Þ kTs /2 G1,Te1 W1 G2,Te2 W2 PNo Figure 6.8 Cascade connection of two noisy two-port devices SYSTEM NOISE 143 The gain of the cascade is described by the product G ¼ G1G2 ð6:50Þ The total output noise consists of three contributions: the noise of the source that is amplified by both systems, the internal noise produced by system 1 and which is amplified by system 2 and the internal noise of system 2. Therefore, the output noise power is   PNo ¼ ðGkTs þ G2G1kTe1 þ G2kTe2 Þ WN 2p ¼ Gk Ts þ Te1 þ Te2 G1 WN 2p ð6:51Þ where the temperature expression Tsys ¼4 Ts þ Te1 þ Te2 G1 ð6:52Þ is called the system noise temperature. From this it follows that the effective noise temperature of the cascade of the two-port devices in Figure 6.8 (see Equation (6.39)) is Te ¼ Te1 þ Te2 G1 ð6:53Þ and the noise figure of the cascade is found by inserting this equation into Equation (6.41), to yield Fs ¼ 1 þ Te1 T0 þ Te2 G1T0 ¼ Fs1 þ Fs2 À G1 1 ð6:54Þ Repeated application of the given method yields the effective noise temperature Te ¼ Te1 þ Te2 G1 þ Te3 G1G2 þ Á Á Á ð6:55Þ and from that the noise figure of a cascade of three or more systems is Fs ¼ 1 þ Te1 T0 þ Te2 G1T0 þ Te3 G1G2T0 þ Á Á Á ¼ Fs1 þ Fs2 À G1 1 þ Fs3 À 1 G1G2 þ Á Á Á ð6:56Þ These two equations are known as the Friis formulas. From these formulas it is concluded that in a cascade connection the first stage plays a crucial role with respect to the noise behaviour; namely the noise from this first stage fully contributes to the output noise, whereas the noise from the next stages is to be reduced by a factor equal to the gain that precedes these stages. Therefore, in designing a system consisting of a cascade, the first stage needs special attention; this stage should show a noise figure that is as low as possible and a gain that is as large as possible. When the gain of the first stage is large, the effective noise temperature and noise figure of the cascade are virtually determined by those of the first stage. Following stages can provide further gain and eventual filtering, but will hardly influence the noise performance of the cascade. This means that the design demands of these stages can be relaxed. 144 NOISE IN NETWORKS AND SYSTEMS Suppose that the first stage is a passive two-port device (e.g. a connection cable) with loss L1. Inserting G1 ¼ 1=L1 into Equation (6.56) yields Fs ¼ L1 þ L1ðFs2 À 1Þ þ L1 Fs3 À G2 1 þ Á Á Á ¼ L1Fs2 þ L1 Fs3 À G2 1 þ Á Á Á ð6:57Þ Such a situation always causes the signal-to-noise ratio to deteriorate severely as the noise figure of the cascade consists mainly of that of the second (amplifier) stage multiplied by the loss of the passive first stage. When the second stage is a low-noise amplifier, this amplifier cannot repair the deterioration introduced by the passive two-port device of the first stage. Therefore, in case a lossy cable is needed to connect a low-noise device to processing equipment, the source first has to be amplified by a low-noise amplifier before applying it to the connection cable. This is elucidated by the next example. Example 6.4: Consider a satellite antenna that is connected to a receiver by means of a coaxial cable and an amplifier. The connection scheme and data of the different components are given in Figure 6.9. The antenna noise is determined by the low effective noise temperature of the dark sky (30 K) and produces an information signal power of À90 dBm in a bandwidth of 1 MHz at the input of the cable, which is at room temperature. All impedances are such that all the time maximum power transfer occurs. The receiver needs at least a signal-to-noise ratio of 17 dB. The question is whether the cascade can meet this requirement. The signal power at the input of the receiver is Psr ¼ ðÀ90 À 2 þ 60Þ dBm ¼ À32 dBm ) 0:63 mW ð6:58Þ Using Equation (6.51), the noise power at the input of the receiver is PN0 ¼ Gampl Gcoax k Tsys WN 2p ð6:59Þ antenna coax amplifier receiver Ts = 30K L = 2 dB G ampl = 60 dB Fs,ampl = 3.5 dB Ps = −90 dBm S/N >17dB Figure 6.9 Satellite receiving circuit SYSTEM NOISE 145 On the linear scale, Gampl ¼ 106 and Gcoax ¼ 0:63. The effective noise temperatures of the coax and amplifier, according to Equation (6.41), are Te;coax ¼ T0ðFs;coax À 1Þ ¼ 290ð1:58 À 1Þ ¼ 168 K Te;ampl ¼ T0ðFs;ampl À 1Þ ¼ 290ð2:24 À 1Þ ¼ 360 K ð6:60Þ Inserting the numerical data into Equation (6.59) yields   PN0 ¼ 106  0:63  1:38  10À23 30 þ 168 þ 360 0:63  106 ¼ 6:7  10À9 ð6:61Þ The ratio of Psr and PN0 produces the signal-to-noise ratio S N ¼ Psr PN0 ¼ 0:63  10À6 6:7  10À9 ¼ 94 ) 19:7 dB ð6:62Þ It is concluded that the cascade satisfies the requirement of a minimum signal-to-noise ratio of 17 dB. & Although the suppliers characterize the components by the noise figure, in calculations as given in this example it is often more convenient to work with the effective noise temperatures in the way shown. Equation (6.41) gives a simple relation between the two data. From the example it is clear that the coaxial cable does indeed cause the noise of the amplifier to be dominant. Example 6.5: As a second example of the noise figure of cascaded systems, we consider two different optical amplifiers, namely the so-called Erbium-doped fibre amplifier (EDFA) and the semiconductor optical amplifier (SOA). The first type is actually a fibre and so the insertion in a fibre link will give small coupling losses, let us say 0.5 dB. The second type, being a semiconductor device, has smaller waveguide dimensions than that of a fibre, which causes relatively high loss, let us say a 3 dB coupling loss. From physical reasoning it follows that optical amplifiers have a minimum noise figure of 3 dB. Let us compare the noise figure when either amplifier is inserted in a fibre link, where each of them has an amplification of 30 dB. On insertion we can distinguish three stages: (1) the coupling from the transmission fibre to the amplifier, (2) the amplifier device itself (EDFA or SOA) and (3) the output coupling from the amplifier device to the fibre. Using Equation (6.57) and the given data of these stages, the noise figure and other relevant data of the insertion are summarized in Table 6.1; note that in this table all data are in dB (see Appendix B). It follows from these data that the noise figure on insertion of the SOA is approximately 2.5 dB worse than that of the EDFA. This is almost completely attributed to the higher coupling loss at the front end of the amplifier. The output coupling hardly influences this number; it only contributes to a lower net gain. & 146 NOISE IN NETWORKS AND SYSTEMS Table 6.1 Comparing different optical amplifiers EDFA SOA Unit Gain of device 30 30 dB Coupling loss ðÂ2Þ 0.5 3 dB Noise figure device 3 3 dB Noise figure on insertion 3.5 6 dB Net insertion gain 29 24 dB These two examples clearly show that in a cascade it is of the utmost importance that the first component (the front end) consists of a low-noise amplifier with a high gain, so that the front end contributes little noise and reduces the noise contribution of the other components in the cascade. 6.5 SUMMARY A stochastic process is called ‘white noise’ if its power spectral density has a constant value for all frequencies. From a physical point of view this is impossible; namely this would imply an infinitely large amount of power. The concept in the first instance is therefore only of mathematical and theoretical value and may probably be used as a model in a limited but practically very wide frequency range. This holds specifically for thermal noise that is produced in resistors. In order to analyse thermal noise in networks and systems, we introduced the The´venin and Norton equivalent circuit models. They consist of an ideal, that is noise-free, resistor in series with a voltage source or in parallel with a current source. Then, using network theoretical methods and the results from Chapter 4, the noise at the output of the network can easily be described. Several resistors in a network are considered as independent noise sources, where the superposition principle may be applied. Therefore, the total output power spectral density consists of the sum of the output spectra due to the individual resistors. Calculating the noise behaviour of systems based on all the noisy components requires detailed data of the constituting components. This leads to lengthy calculations and frequently the detailed data are not available. A way out is offered by noise characterization of (sub)systems based on their output data. These output noise data are usually provided by component suppliers. Important data in this respect are the effective noise temperature and/ or the noise figure. On the basis of these parameters, the influence of the subsystems on the noise performance of a cascade can be calculated. From such an analysis it appears that the first stage of a cascade plays a crucial role. This stage should contribute as little as possible to the output noise (i.e. it must have a low effective noise temperature or, equivalently, a low-noise figure) and a high gain. The use of cables and attenuators as a first stage has to be avoided as they strongly deteriorate the signal-to-noise ratio. Such components should be preceded by low-noise amplifiers with a high gain. 6.6 PROBLEMS 6.1 Consider the thermal noise spectrum given by Equation (6.3). (a) For what values of ! will this given spectrum have a value larger than 0:9  2kTR at room temperature? PROBLEMS 147 (b) Use Matlab to plot this power spectral density as a function of frequency, for R ¼ 1 at room temperature. (c) What is the significance of thermal noise in the optical domain, if it is realized that the optical domain as it is used for optical communication runs to a maximum wavelength of 1650 nm? 6.2 A resistance R1 is at absolute temperature T1 and a second resistance R2 is at absolute temperature T2. (a) What is the equivalent noise temperature of the series connection of these two resistances? (b) If T1 ¼ T2 ¼ T what in that case is the value of Te? 6.3 Answer the same questions as in Problem 6.2 but now for the parallel connection of the two resistances. 6.4 Consider once more the circuit of Problem 6.2. A capacitor with capacitance C1 is connected parallel to R1 and a capacitor with capacitance C2 is connected parallel to R2. (a) Calculate the equivalent noise temperature. (b) Is it possible to select the capacitances such that Te becomes independent of frequency? 6.5 A resistor with a resistance value of R is at temperature T kelvin. A coil is connected parallel to this resistor with a self-inductance L henry. Calculate the mean value of the energy that is stored in the coil as a consequence of thermal noise produced by the resistor. 6.6 An electrical circuit consists of a loop of three elements in series, two resistors and a capacitor. The capacitance is C farad and the resistances are R1 and R2 respectively. Resistance R1 is at temperature T1 K and resistance R2 is at T2 K. Calculate the mean energy stored in the capacitor as a consequence of the thermal noise produced by the resistors. 6.7 A thermal noise source has an internal impedance of Zð!Þ. The noise source is loaded by the load impedance Zlð!Þ. (a) Show that a maximum power transfer from the noise source to the load occurs if Zl ¼ ZÃð!Þ. (b) In that case what is the available power spectral density? 6.8 A resistor with resistance R1 is at absolute temperature T1. A second resistor with resistance R2 is at absolute temperature T2. The resistors R1 and R2 are connected in parallel. (a) What is the spectral density of the net amount of power that is exchanged between the two resistors? (b) Does the colder of the two resistors tend to further cool down due to this effect or heat up? In other words does the system strive for temperature equalization or does it strive to increase the temperature differences? (c) What is the power exchange if the two temperatures are of equal value? 148 NOISE IN NETWORKS AND SYSTEMS R R L C Figure 6.10 R C V(t) A=103 H(ω ) Vo(t) Figure 6.11 6.9 Consider the circuit in Figure 6.10, where all components are at the same temperature. (a) The thermal noise produced by the resistors becomes manifest at the terminals. Suppose that the values of the components are such that the noise spectrum at the terminals is white. Derive the conditions in order for this to happen. (b) In that case what is the impedance at the terminals? 6.10 Consider the circuit presented in Figure 6.11. The input impedance of the amplifier is infinitely high. (a) Derive the expression for the spectral density SVV ð!Þ of the input voltage VðtÞ of the amplifier as a consequence of the thermal noise in the resistance R. The lowpass filter Hð!Þ is ideal, i.e. & Hð!Þ ¼ expðÀj!Þ; j!j W 0; j!j > W In the passband of Hð!Þ the constant  ¼ 1 and the voltage amplification A can also be taken as constant and equal to 103. The amplifier does not produce any noise. The component values of the input circuit are C ¼ 200 nF and R ¼ 1 k . The resistor is at room temperature so that kT ¼ 4  10À21 W s. (b) Calculate the r.m.s. value of the output voltage VoðtÞ of the filter in the case W ¼ 1=ðRCÞ. 6.11 Consider the circuit given in Figure 6.12. The data are as follows: R ¼ 50 ; L ¼ 1 mH; C ¼ 400 pF and A ¼ 100. (a) Calculate the spectral density of the noise voltage at the input of the amplifier as a consequence of the thermal noise produced by the resistors. Assume that these resistors are at room temperature and the other components are noise free. PROBLEMS 149 A FILTER L C delay TD R R Figure 6.12 (b) Calculate the transfer function Hð!Þ from the output amplifier to the input filter. (c) Calculate the spectral density of the noise voltage at the input of the filter. (d) Calculate the r.m.s. value of the noise voltage at the filter output in the case where the filter is ideal lowpass with a transfer of 1 and a cut-off angular frequency !c ¼ p=TD, where TD ¼ 10 ns. 6.12 A signal source has a source impedance of 50 and an equivalent noise temperature of 3000 K. This source is terminated by the input impedance of an amplifier, which is also 50 . The voltage across this resistor is amplified and the amplifier itself is noise free. The voltage transfer function of the amplifier is Að!Þ ¼ 1 100 þ j! where  ¼ 10À8 s. The amplifier is at room temperature. Calculate the variance of the noise voltage at the output of the amplifier. 6.13 An amplifier is constituted from three stages with effective noise temperatures of Te1 ¼ 1300 K; Te2 ¼ 1750 K and Te3 ¼ 2500 K, respectively, and where stage number 1 is the input stage, etc. The power gains amount to G1 ¼ 20, G2 ¼ 10 and G3 ¼ 5, respectively. (a) Calculate the effective noise temperature of this cascade of amplifier stages. (b) Explain why this temperature is considerably lower than Te2, respectively Te3. 6.14 An antenna has an impedance of 300 . The antenna signal is amplified by an amplifier with an input impedance of 50 . In order to match the antenna to the amplifier input impedance a resistor with a resistance of 300 is connected in series with the antenna and parallel to the amplifier input a resistance of 50 is connected. (a) Sketch a block schematic of antenna, matching network and amplifier. (b) Calculate the standard noise figure of the matching network. (c) Do the resistances of 300 and 50 provide matching of the antenna and amplifier? Support your answer by a calculation. (d) Design a network that provides all the matching functionalities. (e) What is the standard noise figure of this latter network? Compare this with the answer found for question (b). 150 NOISE IN NETWORKS AND SYSTEMS 6.15 An antenna is on top of a tall tower and is connected to a receiver at the foot of the tower by means of a cable. However, before applying the signal to the cable it is amplified. The amplifier has a power gain of 20 dB and a noise figure of F ¼ 3 dB. The cable has a loss of 6 dB, while the noise figure of the receiver amounts to 13 dB. All impedances are matched; i.e. between components the maximum power transfer occurs. (a) Calculate the noise figure of the system. (b) Calculate the noise figure of the modified system where the amplifier is placed between the cable and the receiver at the foot of the tower instead of between the antenna and the cable at the top of the tower. 6.16 Reconsider Example 6.4. Interchange the order of the coaxial cable and the amplifier. Calculate the signal-to-noise ratio at the input of the receiver for this new situation. 6.17 An antenna is connected to a receiver via an amplifier and a cable. For proper operation the receiver needs at its input a signal-to-noise ratio of at least 20 dB. The amplifier is directly connected to the antenna and the cable connects the amplifier (power amplification of 60 dB) to the receiver. The cable has a loss of 1 dB and is at room temperature (290 K). The effective noise temperature of the antenna amounts to 50 K. The received signal is À90 dBm at the input of the amplifier and has a bandwidth of 10 MHz. All impedances are such that the maximum power transfer occurs. (a) Present a block schematic of the total system and indicate in that sketch the relevant parameters. (b) Calculate the signal power at the input of the receiver. (c) The system designer can select one out of two suppliers for the amplifier. The suppliers A and B present the data given in Table 6.2. Which of the two amplifiers can be used in the system, i.e. on insertion of the amplifiers in the system which one will meet the requirement for the signal-to-noise ratio? Support your answer with a calculation. 6.18 Consider a source with a real source impedance of Rs. There are two passive networks as given in Figure 6.13. Resistance R1 is at temperature T1 K and resistance R2 is at temperature T2 K. (a) Calculate the available power gain and standard noise factor when the circuit comprising R1 is connected to the source. (b) Calculate the available power gain and standard noise factor when the circuit comprising R2 is connected to the source. Table 6.2 Noise figure F (dB) S=N reduction at the source temperature of 120 K (dB) Supplier A B 3.5 – – 6 R 1,T1 Figure 6.13 PROBLEMS 151 R2, T2 C Figure 6.14 (c) Now assume that the two networks are cascaded where R1 is connected to the source and R2 to the output. Calculate the available gain and the standard noise figure of the cascade when connected to this source. (d) Do the gains and the noise figures satisfy Equations (6.50) and (6.54), respectively? Explain your conclusion. (e) Redo the calculations of the gain and noise figure when Rs þ R1 is taken as the source impedance for the second two-port device, i.e. the impedance of the source and the first two-port device as seen from the viewpoint of the second two-port device. (f) Do the gains and the noise figures in case (e) satisfy Equations (6.50) and (6.54), respectively? 6.19 Consider a source with a complex source impedance of Zs. This source is loaded by the passive network given in Figure 6.14. (a) Calculate the available power gain and noise factor of the two-port device when it is connected to the source. (b) Do the answers from (a) surprise you? If the answer is ‘yes’ explain why. If the answer is ‘no’ explain why not. 6.20 Consider the circuit given in Figure 6.15. This is a so-called ‘constant resistance network’. (a) Show that the input impedance of this circuit equals R0 if Z1 Z2 ¼ R20 and the circuit is terminated by a resistance R0. (b) Calculate the available power gain and noise figure of the circuit (at temperature T kelvin) if the source impedance equals R0. 152 NOISE IN NETWORKS AND SYSTEMS Z1 R0 R0 Z2 Figure 6.15 (c) Suppose that two of these circuits at different temperatures and with different gains are put in cascade and that the source impedance equals R0 once more. Calculate the overall available power gain and noise figure. (d) Does the overall gain equal the product of the gains? (e) Under what circumstances does the noise figure obey Equation (6.54)? 7 Detection and Optimal Filtering Thus far the treatment has focused on the description of random signals and their analyses, and how these signals are transformed by linear time-invariant systems. In this chapter we take a somewhat different approach; namely starting with what is known about input processes and of system requirements we look for an optimum system. This means that we are going to perform system synthesis. The approach achieves an optimal reception of information signals that are corrupted by noise. In this case the input process consists of two parts, the information bearing or data signal and noise, and we may wonder what the optimal receiver or processing looks like, subject to some criterion. When designing an optimal system three items play a crucial role. These are: 1. A description of the input noise process and the information bearing signal; 2. Conditions to be imposed on the system; 3. A criterion that defines optimality. In the following we briefly comment on these items: 1. It is important to know the properties of the system inputs, e.g. the power spectral density of the input noise, whether it is wide-sense stationary, etc. What does the information signal look like? Are information signal and noise additive or not? 2. The conditions to be imposed on the system may influence performance of the receiver or the processing. We may require the system to be linear, time-invariant, realizable, etc. To start with and to simplify matters we will not bother about realizability. In specific cases it can easily be included. 3. The criterion will depend on the problem at hand. In the first instance we will consider two different criteria, namely the minimum probability of error in detecting data signals and the maximum signal-to-noise ratio. These criteria lead to an optimal linear filter called the matched filter. This name will become clear in the sequel. Although the criteria are quite different, we will show that there is a certain relationship in specific cases. In a third approach we will look for a filter that produces an optimum estimate of the Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 154 DETECTION AND OPTIMAL FILTERING realization of a stochastic process, which comes along with additive noise. In such a case we use the minimum mean-squared error criterion and end up with the so-called Wiener filter. 7.1 SIGNAL DETECTION 7.1.1 Binary Signals in Noise Let us consider the transmission of a known deterministic signal that is disturbed by noise; the noise is assumed to be additive. This situation occurs in a digital communication system where, during successive intervals of duration T seconds, a pulse of known shape may arrive at the receiver (see the random data signal in Section 4.5). In such an interval the pulse has been sent or not. In accordance with Section 4.5 this transmitted random data signal is denoted by X ZðtÞ ¼ An pðt À nTÞ n ð7:1Þ Here An is randomly chosen from the set f0; 1g. The received signal is disturbed by additive noise. The presence of the pulse corresponds to the transmission of a binary digit ‘1’ ðAn ¼ 1Þ, whereas absence of the pulse in a specific interval represents the transmission of a binary digit ‘0’ ðAn ¼ 0Þ. The noise is assumed to be stationary and may originate from disturbance of the channel or has been produced in the front end of the receiver equipment. Every T seconds the receiver has to decide whether a binary ‘1’ or a binary ‘0’ has been sent. This decision process is called detection. Noise hampers detection and causes errors to occur in the detection process, i.e. ‘1’s may be interpreted as ‘0’s and vice versa. During each bit interval there are two possible mutually exclusive situations, called hypotheses, with respect to the received signal RðtÞ: H0: RðtÞ ¼ NðtÞ; 0tT H1: RðtÞ ¼ pðtÞ þ NðtÞ; 0 t T ð7:2Þ ð7:3Þ The hypothesis H0 corresponds to the situation that a ‘0’ has been sent ðAn ¼ 0Þ. In this case the received signal consists only of the noise process NðtÞ. Hypothesis H1 corresponds to the event that a ‘1’ has been sent ðAn ¼ 1Þ. Now the received signal comprises the known pulse shape pðtÞ and the additive noise process NðtÞ. It is assumed that each bit occupies the ð0; TÞ interval. Our goal is to design the receiver such that in the detection process the probability of making wrong decisions is minimized. If the receiver decides in favour of hypothesis H0 and it produces a ‘0’, we denote the estimate of An by A^n and say that A^n ¼ 0. In case the receiver decides in favour of hypothesis H1 and a ‘1’ is produced, we denote A^n ¼ 1. Thus the detected bit A^n 2 f0; 1g. In the detection process two types of errors can be made. Firstly, the receiver decides in favour of hypothesis H1, i.e. a ‘1’ is detected ðA^n ¼ 1Þ, whereas a ‘0’ has been sent ðAn ¼ 0Þ. The conditional probability of this event is PðA^n ¼ 1j H0Þ ¼ PðA^n ¼ 1j An ¼ 0Þ. Secondly, the receiver decides in favour of hypothesis H0 ðA^n ¼ 0Þ, whereas a ‘1’ has been sent ðAn ¼ 1Þ. The conditional probability of this event is PðA^n ¼ 0 j H1Þ ¼ PðA^n ¼ 0 j An ¼ 1Þ. In a long sequence of transmitted bits the prior SIGNAL DETECTION 155 probability of sending a ‘0’ is given by P0 and the prior probability of a ‘1’ by P1. We assume that these probabilities are known in the receiver. In accordance with the law of total probability the bit error probability is given by Pe ¼ P0 PðA^n ¼ 1j H0Þ þ P1 PðA^n ¼ 0 j H1Þ ¼ P0 PðA^n ¼ 1j An ¼ 0Þ þ P1 PðA^n ¼ 0 j An ¼ 1Þ ð7:4Þ This error probability is minimized if the receiver chooses the hypothesis with the highest conditional probability, given the process RðtÞ. It will be clear that the conditional probabilities of Equation (7.4) depend on the signal pðtÞ, the statistical properties of the noise NðtÞ and the way the receiver processes the received signal RðtÞ. As far as the latter is concerned, we assume that the receiver converts the received signal RðtÞ into K numbers (random variables), which are denoted by the K-dimensional random vector r ¼ ðr1; r2; . . . ; rKÞ ð7:5Þ The receiver chooses the hypothesis H1 if PðH1j rÞ ! PðH0j rÞ, or equivalently P1 frðr j H1Þ ! P0 frðr j H0Þ, since it follows from Bayes’ theorem (reference [14]) that PðHi j rÞ ¼ Pi frðr j Hi frðrÞ Þ ; i ¼ 0; 1 ð7:6Þ From this it follows that the decision can be based on the so-called likelihood ratio ÃðrÞ ¼4 frðr frðr j j H1Þ H0Þ H><1 H0 Ã0 ¼4 P0 P1 ð7:7Þ In other words, hypothesis H1 is chosen if ÃðrÞ > Ã0 and hypothesis H0 is chosen if ÃðrÞ < Ã0. The quantity Ã0 is called the decision threshold. In taking the decision the receiver partitions the vector space spanned by r into two parts, R0 and R1, called the decision regions. The boundary between these two regions is determined by Ã0. In the region R0 we have the relation ÃðrÞ < Ã0 and an observation of r in this region causes the receiver to decide that a binary ‘0’ has been sent. An observation in the region R1, i.e. ÃðrÞ ! Ã0, makes the receiver decide that a binary ‘1’ has been sent. The task of the receiver therefore is to transform the received signal RðtÞ into the random vector r and determine to which of the regions R0 or R1 it belongs. Later we will go into more detail of this signal processing. Example 7.1: Consider the two conditional probability densities fr ðr j H0Þ ¼ p1ffiffiffiffiffi 2p  exp À r2 2 fr ðr j H1Þ ¼ 1 2 expðÀj rj Þ and the prior probabilities P0 ¼ P1 ¼ 1 2 ð7:8Þ ð7:9Þ ð7:10Þ 156 DETECTION AND OPTIMAL FILTERING fr(r |H1) fr(r |H0) R1 R0 R1 R0 R1 r Figure 7.1 Conditional probability density functions of the example and the decision regions R0 and R1 Let us calculate the decision regions for this situation. By virtue of Equation (7.10) the decision threshold is set to one and the decision regions are found by equating the right-hand sides of Equations (7.8) and (7.9): p1ffiffiffiffiffi  exp À r2 ¼ 1 expðÀj rj Þ 2p 22 ð7:11Þ The two expressions are depicted in Figure 7.1. As seen from the figure, the functions are even symmetric and, confining to positive values of r, this equation can be rewritten as the quadratic r2 À 2r À 2 ln p2ffiffiffiffiffi ¼ 0 2p ð7:12Þ Solving this yields the roots r1 ¼ 0:259 and r2 ¼ 1:741. Considering negative r values produces the same negative values for the roots. Hence it may be concluded that the decision regions are described by R0 ¼ fr : 0:259 < j rj < 1:741g R1 ¼ fr : ðj rj < 0:259Þ [ ðj rj > 1:741Þ ð7:13Þ ð7:14Þ & One may wonder what to do when an observation is exactly at the boundaries of the decision regions. An arbitrary decision can be made, since the probability of this event approaches zero. SIGNAL DETECTION 157 The conditional error probabilities in Equation (7.4) are written as ZZ PðA^n ¼ 1j H0Þ ¼ PfÃðrÞ ! Ã0 j H0g ¼ . . . frðr j H0Þ dr1 Á Á Á drK Z R1 Z PðA^n ¼ 0 j H1Þ ¼ PfÃðrÞ < Ã0 j H1g ¼ . . . frðr j H1Þ dr1 Á Á Á drK R0 ð7:15Þ ð7:16Þ The minimum total bit error probability is found by inserting these quantities in Equation (7.4). Example 7.2: An example of a received data signal (see Equation (7.1)) has been depicted in Figure 7.2(a). Let us assume that the signal RðtÞ is characterized by a single number r instead of a vector and that in the absence of noise ðNðtÞ  0Þ this number is symbolically denoted by ‘0’ (in the case of hypothesis H0Þ or ‘1’ (in the case of hypothesis H1Þ. Furthermore, assume that in the presence of noise NðtÞ a stochastic Gaussian variable should be added to this characteristic number. For this situation the conditional probability density functions are given in Figure 7.2(a) upper right. In Figure 7.2(b) these functions are depicted once more, but now in a somewhat different way. The boundary that separates the decision regions R0 and R1 reduces to a single point. This point r0 is determined by Ã0. The bit error probability is now written as Z1 Z r0 Pe ¼ P0 frðr j H0Þ dr þ P1 frðr j H1Þ dr r0 À1 ð7:17Þ P1fr (r |H1) "1" "0" -T 0 T 2T (a) R0 t R1 P0fr (r |H0) P0fr(r |H0) P1fr (r |H1) 0 r0 1 r (b) Figure 7.2 (a) A data signal disturbed by Gaussain noise and (b) the corresponding weighted (by the prior probabilities) conditional probability density functions and the decision regions R0 and R1 158 DETECTION AND OPTIMAL FILTERING The first term on the right-hand side of this equation is represented by the right shaded region in Figure 7.2(b) and the second term by the left shaded region in this figure. The threshold value r0 is to be determined such that Pe is minimized. To that end Pe is differentiated with respect to r0 dPe dr0 ¼ ÀP0 fr ðr0 j H0Þ þ P1 fr ðr0 j H1Þ ð7:18Þ When this expression is set equal to zero we once again arrive at Equation (7.7); in this way this equation has been deduced in an alternative manner. Now it appears that the optimum threshold value r0 is found at the intersection point of the curves P0 frðr j H0Þ and P1 frðr j H1Þ. If the probabilities P0 and P1 change but the probability density function of the noise NðtÞ remains the same, then the optimum threshold value shifts in the direction of the binary level that corresponds to the shrinking prior probability. Remembering that we considered the case of Gaussian noise, it is concluded that the integrals in Equation (7.17) can be expressed using the well-known Q function (see Appendix F), which is defined as QðxÞ ¼4 Z p1ffiffiffiffiffi 1  exp À y2 dy 2p x 2 ð7:19Þ This function is related to the erfc(Á) function as follows:  QðxÞ ¼ 1 erfc pxffiffi 2 2 ð7:20Þ Both functions are tabulated in many books or can be evaluated using software packages. They are presented graphically in Appendix F. More details on the Gaussian noise case are presented in the next section. & 7.1.2 Detection of Binary Signals in White Gaussian Noise In this subsection we will assume that in the detection process as described in the foregoing the disturbing noise NðtÞ has a Gaussian probability density function and a white spectrum with a spectral density of N0=2. This latter assumption means that filtering has to be performed in the receiver. This is understood if we realize that a white spectrum implies an infinitely large noise variance, which leads to problems in the integrals that appear in Equation (7.17). Filtering limits the extent of the noise spectrum to a finite frequency band, thereby limiting the noise variance to finite values and thus making the integrals well defined. The received signal RðtÞ is processed in the receiver to produce the vector r in a signal space f’kðtÞg that completely describes the signal pðtÞ and is assumed to be an orthonormal set (see Appendix A) ZT rk ¼ kðtÞ RðtÞ dt; k ¼ 1; . . . ; K 0 ð7:21Þ SIGNAL DETECTION 159 As the operation given by Equation (7.21) is a linear one, it can be applied to the two terms of Equation (7.3) separately, so that rk ¼ pk þ nk; k ¼ 1; . . . ; K ð7:22Þ with ZT pk ¼4 kðtÞ pðtÞ dt; k ¼ 1; . . . ; K 0 ð7:23Þ and ZT nk ¼4 kðtÞ NðtÞ dt; k ¼ 1; . . . ; K 0 ð7:24Þ In fact, the processing in the receiver converts the received signal RðtÞ into a vector r that consists of the sum of the deterministic signal vector p, of which the elements are given by Equation (7.23), and the noise vector n, of which the elements are given by Equation (7.24). As NðtÞ has been assumed to be Gaussian, the random variables nk will be Gaussian as well. This is due to the fact that when a linear operation is performed on a Gaussian variable the Gaussian character of the random variable is maintained. It follows from Appendix A that the elements of the noise vector are orthogonal and all of them have the same variance N0=2. In fact, the noise vector n defines the relevant noise (Appendix A and reference [14]). Considering the case of binary detection, the conditional probability density functions for the two hypotheses are now ! frðr j H0Þ ¼ qffiffiffiffi1ffiffiffiffiffiffiffiffiffiffiffi exp ðp N0ÞK À 1 N0 XK k¼1 rk2 ð7:25Þ and ! frðr j H1Þ ¼ qffiffiffiffi1ffiffiffiffiffiffiffiffiffiffiffi exp ðp N0ÞK À 1 N0 XK ðrk k¼1 À pk Þ2 ð7:26Þ Using Equation (7.7), the likelihood ratio is written as " # ÃðrÞ ¼ exp À 1 N0 XK ðrk k¼1 À pk Þ2 þ 1 N0 XK k¼1 r 2 k ¼ exp ! 2 N0 XK k¼1 pk rk À 1 N0 XK k¼1 p 2 k ð7:27Þ In Appendix A it is shown that the term P p2k represents the energy Ep of the deterministic signal pðtÞ. This quantity is supposed to be known at the receiver, so that the only quantity that depends on the transmitted signal consists of the summation over pkrk. By means of 160 DETECTION AND OPTIMAL FILTERING signal processing on the received signal RðtÞ, the value of this latter summation should be determined. This result represents a sufficient statistic [3,9] for detecting the transmitted data in an optimal way. A statistic is an operation on an observation, which is presented by a function or functional. A statistic is said to be sufficient if it preserves all information that is relevant for estimating the data. In this case it means that in dealing with the noise component of r the irrelevant noise components may be ignored (see Appendix A or reference [14]). This is shown as follows: X X pkrk ¼ pkðpk þ nkÞ k Zk T X ¼ kðtÞpðtÞðpk þ nkÞ dt 0 ZT k X ¼ pðtÞ ðpk þ nkÞkðtÞ dt 0 ZT k ¼ pðtÞ½pðtÞ þ Nrðtފ dt 0 ð7:28Þ where NrðtÞ is the relevant noise part of NðtÞ (see Appendix A or reference [14]). Since the irrelevant part of the noise NiðtÞ is orthogonal to the signal space (see Appendix A), adding this part of the noise to the relevant noise in the latter expression does not influence the result of the integration: X ZT pkrk ¼ pðtÞ½pðtÞ þ NrðtÞ þ Niðtފ dt k 0 ZT ZT ¼ pðtÞ½pðtÞ þ Nðtފ dt ¼ pðtÞRðtÞ dt 0 0 ð7:29Þ The implementation of this operation is as follows. The received signal RðtÞ is applied to a linear, time-invariant filter with the impulse response pðT À tÞ. The output of this filter is sampled at the end of the bit interval (at t0 ¼ T), and this sample value yields the statistic of Equation (7.29). This is a simple consequence of the convolution integral. The output signal of the filter is denoted by YðtÞ, so that ZT ZT YðtÞ ¼ RðtÞ Ã hðtÞ ¼ RðtÞ Ã pðT À tÞ ¼ RðÞhðt À Þ d ¼ RðÞpð À t þ TÞ d 0 0 ð7:30Þ At the sampling instant t0 ¼ T the value of the signal at the output is ZT YðTÞ ¼ RðÞpðÞ d 0 ð7:31Þ The detection process proceeds as indicated in Section 7.1.1; i.e. the sample value YðTÞ is compared to the threshold value. This threshold value D is found from Equations (7.27) and R(t ) matched filter SIGNAL DETECTION 161 closed at t 0 =T decision An device threshold D Figure 7.3 Optimal detector for binary signals (7.7), and is implicitly determined by   Ã0 ¼ exp 2D À Ep N0 ð7:32Þ Hypothesis H0 is chosen whenever YðTÞ < D, whereas H1 is chosen whenever YðTÞ ! D. In the special binary case where P0 ¼ P1 ¼ 12, it follows that D ¼ Ep=2. Note that in the case at hand the signal space will be one-dimensional. The filter with the impulse response hðtÞ ¼ pðT À tÞ is called a matched filter, since the shape of its impulse response is matched to the pulse pðtÞ. The scheme of the detector is very simple; namely the signal is filtered by the matched filter and the output of this filter is sampled at the instant t0 ¼ T. If the sampled value is smaller than D then the detected bit is A^n ¼ 0 ðH0Þ and if the sampled value is larger than D then the receiver decides A^n ¼ 1 ðH1Þ. This is represented schematically in Figure 7.3. 7.1.3 Detection of M-ary Signals in White Gaussian Noise The situation of M-ary transmission is a generalization of the binary case. Instead of two different hypotheses and corresponding signals there are M different hypotheses, defined as H0 : RðtÞ ¼ p0ðtÞ þ NðtÞ H1 : RðtÞ ¼ p1ðtÞ þ NðtÞ Á ÁÁÁ Hi : RðtÞ ¼ piðtÞ þ NðtÞ Á ÁÁÁ HM : RðtÞ ¼ pMðtÞ þ NðtÞ ð7:33Þ As an example of this situation we mention FSK; in binary FSK we have M ¼ 2. To deal with the M-ary detection problem we do not use the likelihood ratio directly; in order to choose the maximum likely hypothesis we take a different approach. We turn to our fundamental criterion; namely the detector chooses the hypothesis that is most probable, given the received signal. The probabilities of the different hypotheses, given the received signal, are given by Equation (7.6). When selecting the hypothesis with the highest probability, the denominator frðrÞ may be ignored since it is common for all hypotheses. We are therefore looking for the hypothesis Hi, for which Pi frðr j HiÞ attains a maximum. 162 DETECTION AND OPTIMAL FILTERING For a Gaussian noise probability density function this latter quantity is " # Pi frðr j HiÞ ¼ Pi qffiffiffiffi1ffiffiffiffiffiffiffiffiffiffiffi exp ðp N0ÞK À 1 N0 XK ðrk k¼1 À pk;iÞ2 ; i ¼ 1; . . . ; M ð7:34Þ with pk;i the kth element of piðtÞ in the signal space; the summation over k actually represents the distance in signal space between the received signal and piðtÞ and is called the distance metric. Since Equation (7.34) is a monotone-increasing function of r, the decision may also be based on the selection of the largest value of the logarithm of expression (7.34). This means that we compare the different values of ln½Pi frðr j Hiފ ¼ ln Pi À 1 N0 XK ðrk2 k¼1 þ p2k;i À 2rk pk;i Þ À ln qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðp N0ÞK ¼ ln Pi À 1 N0 XK k¼1 rk2 À 1 N0 XK k¼1 p2k;i þ 2 N0 XK k¼1 rk pk;i À ln qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðp N0ÞK ; i ¼ 1; . . . ; M ð7:35Þ The second and fifth term on the right-hand side are common for all hypotheses, i.e. they do not depend on i, and thus they may be ignored in the decision process. We finally end up with the decision statistics di ¼ N0 2 ln Pi À Ei 2 þ XK rk pk;i ; k¼1 i ¼ 1; . . . ; M ð7:36Þ where Ei is the energy in the signal piðtÞ (see Appendix A) and Equation (7.35) has been multiplied by N0=2, which is allowed since it is a constant and does not influence the decision. For ease of notation we define bi ¼4 N0 2 ln Pi À Ei 2 ð7:37Þ so that XK di ¼ bi þ rkpk;i k¼1 ð7:38Þ Based on Equations (7.37) and (7.38) we can construct the optimum detector. It is shown in Figure 7.4. The received signal is filtered by a bank of matched filters, the ith filter being matched to the signal piðtÞ. The outputs of the filters are sampled and the result represents the last term of Equation (7.38). Next, the bias terms bi given by Equation (7.37) are added to these outputs, as indicated in the figure. The resulting values di are applied to a circuit that selects the largest, thereby producing the detected symbol A^. This symbol is taken from the alphabet fA1; . . . ; AMg, the same set of symbols from which the transmitter selected its symbols. SIGNAL DETECTION 163 filter matched to p0(t ) closed at t0=T b1 + d1 … closed at bi R(t ) filter matched to pi (t ) t0 =T di select + largest A … filter matched to p (t ) M closed at t0=T bM dM + Figure 7.4 Optimal detector for M-ary signals where the symbols are mapped to different signals fr(r |H0) fr(r |H1) Ed 0 Ed r 2 2 Figure 7.5 Conditional probability density functions in the binary case with the error probability indicated by the shaded area In all these operations it is assumed that the shapes of the several signals piðtÞ as well as their energy contents are known and fixed, and the prior probabilities Pi are known. It is evident that the bias terms may be omitted in case all prior probabilities are the same and all signals piðtÞ carry the same energy. Example 7.3: As an example let us consider the detection of linearly independent binary signals p0ðtÞ and p1ðtÞ in white Gaussian noise. The signal space to describe this signal set is twodimensional. However, it can be reduced to a one-dimensional signal space by converting to a simplex signal spetffiffiffi(ffisffi ee Section A.5). The basis of this signal space is given by ðtÞ ¼ ½p0ðtÞ À p1ðtފ= Ed, where Ed is thpe ffiffieffiffinffi ergy in ptheffiffiffiffiffidifference signal. The signal constellation is given bpy ffitffiffihffiffie coordinates ½À Ed=2Š and ½ Ed=2Š and the distance between the signals amounts to Ed. Superimposed on these signals is the noise with variance N0=2 (see Appendix A, Equation (A.26)). We assume that the two hypotheses are equiprobable. The situation has been depicted in Figure 7.5. In this figure the two conditional probability 164 DETECTION AND OPTIMAL FILTERING density functions are presented. The error probability follows from Equation (7.17), and since the prior probabilities are equal we can conclude that Z1 Pe ¼ frðr j H0Þ dr 0 ð7:39Þ In the figure the value of the error probability is indicated by the shaded area. Since the noise has been assumed to be Gaussian, this probability is written as Pe ¼ Z 1 rffiffi1ffiffiffiffiffiffiffiffiffi 0 2p N0 2 exp6664À  x ÀNp02ffiEffiffidffiffi237775 dx 2 ð7:40Þ Introducing the change of integration variable pffiffiffiffiffi z ¼4 x À Ed rffiffiffi2ffiffi ¼) dx ¼ rffiffiffiffiffi N0 dz N0 2 2 the error probability is written as Pe ¼ p1ffiffiffiffiffi 2p Z1 pffiffiffiffiffiffiffiffiffiffiffiffiffi Ed =ð2N0 Þ  exp À z2 2 dz ð7:41Þ ð7:42Þ This expression is recognized as the well-known Q function (see Equation (7.19) and Appendix F). Finally, the error probability can be denoted as rffiffiffiffiffiffiffiffi Pe ¼ Q Ed 2N0 ð7:43Þ This is a rather general result that can be used for different binary transmission schemes, both for baseband and modulated signal formats. The conditions are that the noise is white, additive and Gaussian, and the prior probabilities are equal. It is concluded that the error probability depends neither on the specific shapes of the received pulses nor on the signal set that has been chosen for the analysis, but only on the energy of the difference between the two pulses. Moreover, the error probability depends on the ratio Ed=N0; this ratio can be interpreted as a signal-to-noise ratio, often expressed in dB (see Appendix B). In signal space the quantity Ed is interpreted as the squared distance of the signal points. The further the signals are apart in the signal space, the lower the error probability will be. This specific example describes a situation that is often met in practice. Despite the fact that we have two linearly independent signals it suffices to provide the receiver with a single matched filter, namely a filter matched to the difference p0ðtÞ À p1ðtÞ, being the basis of the simplex signal set. & FILTERS THAT MAXIMIZE THE SIGNAL-TO-NOISE RATIO 165 7.1.4 Decision Rules 1. Maximum a posteriori probability (MAP) criterion. Thus far the decision process was determined by the so-called posterior probabilities given by Equation (7.6). Therefore this rule is referred to as the maximum a posteriori probability (MAP) criterion. 2. Maximum-likelihood (ML) criterion. In order to apply the MAP criterion the prior probabilities should be known at the receiver. However, this is not always the case. In the absence of this knowledge it may be assumed that the prior probabilities for all the M signals are equal. A receiver based on this criterion is called a maximum-likelihood receiver. 3. The Bayes criterion. In our treatment we have considered the detection of binary data. In general, for signal detection a slightly different approach is used. The basics remain the same but the decision rules are different. This is due to the fact that in general the different detection probabilities are connected to certain costs. These costs are presented in a cost matrix   C¼ C00 C10 C01 C11 ð7:44Þ where Cij is the cost of Hi being detected when actually Hj is transmitted. In radar hypothesis H1 corresponds to a target, whereas hypothesis H0 corresponds to the absence of a target. Detecting a target when actually no target is present is called a false alarm, whereas detecting no target when actually one is there is called a miss. One can imagine that taking action on these mistakes can have severe consequences, which are differently weighed for the two different errors. The detection process can actually have four different outcomes, each of them associated with its own conditional probability. When applying the Bayes criterion the four different probabilities are multiplied by their corresponding cost factors, given by Equation (7.44). This results in the mean risk. The Bayes criterion minimizes this mean risk. For more details see reference [15]. 4. The minimax criterion. The Bayes criterion uses the prior probabilities for minimizing the mean cost. When the detection process is based on wrong assumptions in this respect, the actual cost can be considerably higher than expected. When the probabilities are not known a good strategy is to minimize the maximum cost; i.e. whatever the prior probabilities in practice are, the mean cost can be guaranteed not to be larger than a certain value that can be calculated in advance. For further information on this subject see reference [15]. 5. The Neyman–Pearson criterion. In radar detection the prior probabilities are often difficult to determine. In such situations it is meaningful to invoke the Neyman–Pearson criterion [15]. It maximizes the probability of detecting a target at a fixed false alarm probability. This criterion is widely used in radar detection. 7.2 FILTERS THAT MAXIMIZE THE SIGNAL-TO-NOISE RATIO In this section we will derive a linear time-invariant filter that maximizes the signal-to-noise ratio when a known deterministic signal xðtÞ is received and which is disturbed by additive 166 DETECTION AND OPTIMAL FILTERING noise. This maximum of the signal-to-noise ratio occurs at a specific, predetermined instant in time, the sampling instant. The noise need not be necessarily white or Gaussian. As we assumed in earlier sections, we will only assume it to be wide-sense stationary. The probability density function of the noise and its spectrum are allowed to have arbitrary shapes, provided they obey the conditions to be fulfilled for these specific functions. Let us assume that the known deterministic signal may be Fourier transformed. The value of the output signal of the filter at the sampling instant t0 is yðt0Þ ¼ 1 2p Z1 À1 Xð!ÞHð!Þ expð j!t0Þ d! ð7:45Þ where Hð!Þ is the transfer function of the filter. Since the noise is supposed to be widesense stationary it follows from Equation (4.28) that the power of the noise output of the filter is PN0 ¼ E½N02ðtފ ¼ 1Z1 2p À1 SNN ð!Þj Hð!Þj 2 d! ð7:46Þ with SNNð!Þ the spectrum of the input noise. The output signal power at the sampling instant is achieved by squaring Equation (7.45). Our goal is to find a value of Hð!Þ such that a maximum occurs for the signal-to-noise ratio defined as S N ¼4 j yðt0Þj 2 PN0 ¼  1 2p R21Àp11RÀ1X1ð!SNÞHN ðð!!ÞÞj expð j!t0Þ d! Hð!Þj 2 d! 2 ð7:47Þ For this purpose we use the inequality of Schwarz. This inequality reads  Z 1 Að!ÞBð!Þ d!2 À1 Z1 Z1 j Að!Þj 2 d! j Bð!Þj 2 d! À1 À1 ð7:48Þ The equality holds if Bð!Þ is proportional to the complex conjugate of Að!Þ, i.e. if Að!Þ ¼ C BÃð!Þ ð7:49Þ where C is an arbitrary real constant. With the substitutions pffiffiffiffiffiffiffiffiffiffiffiffiffiffi Að!Þ ¼ Hð!Þ SNN ð!Þ Bð!Þ ¼ Xð!pÞ effixffiffipffiffiffiðffiffijffi!ffiffiffitffi0ffi Þ 2p SNNð!Þ ð7:50Þ ð7:51Þ Equation (7.48) becomes  1 2p Z1 À1 Xð!ÞH ð!Þ expðj!t0 Þ d!2 1Z1 2p À1 SNN ð!Þj Hð!Þj 2 d! 1Z1 2p À1 j Xð!Þj 2 SNN ð!Þ d! ð7:52Þ FILTERS THAT MAXIMIZE THE SIGNAL-TO-NOISE RATIO 167 From Equations (7.47) and (7.52) it follows that S N 1Z1 2p À1 j Xð!Þj 2 SNN ð!Þ d! ð7:53Þ It can be seen that in Equation (7.53) the equality holds if Equation (7.49) is satisfied. This means that the signal-to-noise ratio achieves its maximum value. From the sequel it will become clear that for the special case of white Gaussian noise the filter that maximizes the signal-to-noise ratio is the same as the matched filter that was derived in Section 7.1.2. This name is also used in the generalized case we are dealing with here. From Equations (7.49), (7.50) and (7.51) the following theorem holds. Theorem 12 The matched filter has the transfer function (frequency domain description) Hoptð!Þ ¼ XÃð!Þ SNN ð!Þ expðÀj!t0Þ ð7:54Þ with Xð!Þ the Fourier transform of the input signal xðtÞ, SNNð!Þ the power spectral density function of the additive noise and t0 the sampling instant. We choose the constant C equal to 2p. The transfer function of the optimal filter appears to be proportional to the complex conjugate of the amplitude spectrum of the received signal xðtÞ. Furthermore, Hoptð!Þ appears to be inversely proportional to the noise spectral density function. It is easily verified that an arbitrary value for the constant C may be chosen. From Equation (7.47) it follows that a constant factor in Hð!Þ does not affect the signal-to-noise ratio. In other words, in Hoptð!Þ an arbitrary constant attenuation or gain may be inserted. The sampling instant t0 does not affect the amplitude of Hoptð!Þ but only the phase expðÀj!t0Þ. In the time domain this means a delay over t0. The value of t0 may, as a rule, be chosen arbitrarily by the system designer and in this way may be used to guarantee a condition for realizability, namely causality. The result we derived has a general validity; this means that it is also valid for white noise. In that case we make the substitution SNNð!Þ ¼ N0=2. Once more, choosing a proper value for the constant C, we arrive at the following transfer function of the optimal filter: Hoptð!Þ ¼ XÃð!Þ expðÀj!t0Þ ð7:55Þ This expression is easily transformed to the time domain. Theorem 13 The matched filter for the signal xðtÞ in white additive noise has the impulse response (time domain description) with t0 the sampling instant. hoptðtÞ ¼ xðt0 À tÞ ð7:56Þ 168 DETECTION AND OPTIMAL FILTERING From Theorem 13 it follows that the impulse response of the optimal filter is found by shifting the input signal by t0 to the left over the time axis and mirroring it with respect to t ¼ 0. This time domain description offers the opportunity to guarantee causality by setting hðtÞ ¼ 0 for t < 0. Comparing the result of Equation (7.56) with the optimum filter found in Section 7.1.2, it is concluded that in both situations the optimal filters show the same impulse response. This may not surprise us, since in the case of Gaussian noise the maximum signal-to-noise ratio implies a minimum probability of error. From this we can conclude that the matched filter concept has a broader application than the considerations given in Section 7.1.2. Once the impulse response of the optimal filter is known, the output response of this filter to the input signal xðtÞ can be calculated. This is obtained by applying the well-known convolution integral Z1 Z1 yðtÞ ¼ hoptðÞxðt À Þ d ¼ xðt0 À Þxðt À Þ d À1 À1 ð7:57Þ At the decision instant t0 the value of the output signal yðt0Þ equals the energy of the incoming signal till the moment t0, multiplied by an arbitrary constant that may be introduced in hoptðtÞ. The noise power at the output of the matched filter is PN0 ¼ 1 2p N0 2 Z1 À1 j Hoptð!Þj 2 d! ¼ N0 2 Z1 À1 h2optðtÞ dt ð7:58Þ The last equality in this equation follows from Parseval’s formula (see Appendix G or references [7] and [10]). However, since we found that the impulse response of the optimal filter is simply a mirrored version in time of the received signal (see Equation (7.56)) it is concluded that PN0 ¼ N0 2 Ex ð7:59Þ with Ex the energy content of the signal xðtÞ. From Equations (7.57) and (7.59) the signal-tonoise ratio at the output of the filter can be deduced. Theorem 14 The signal-to-noise ratio at the output of the matched filter at the sampling instant is  S ¼4 j yðt0Þj 2 ¼ 2Ex N max PN0 N0 ð7:60Þ with Ex the energy content of the received signal and N0=2 the spectral density of the additive white noise. Although a method exists to generalize the theory of Sections 7.1.2 and 7.1.3 to include coloured noise, we will present a simpler alternative here. This alternative reduces the problem of coloured noise to that of white noise, for which we now know the solution, as FILTERS THAT MAXIMIZE THE SIGNAL-TO-NOISE RATIO 169 p(t )+N(t ) coloured noise H1(ω) p1(t )+N1(t ) white noise H2(ω) po(t )+No(t ) Figure 7.6 Matched filter for coloured noise presented in the last paragraph. The basic idea is to insert a filter between the input and matched filter. The transfer function of this inserted filter is chosen such that the coloured input noise is transformed into white noise. The receiving filter scheme is as shown in Figure 7.6. It depicts the situation for hypothesis H1, with the input pðtÞ þ NðtÞ. The spectrum of NðtÞ is assumed to be coloured. Based on what we want to achieve, the transfer function of the first filter should satisfy j H1ð!Þj 2 ¼ 1 SNN ð!Þ ð7:61Þ By means of Equation (4.27) it is readily seen that the noise N1ðtÞ at the output of this filter has a white spectral density. For this reason the filter is called a whitening filter. The spectrum of the signal p1ðtÞ at the output of this filter can be written as P1ð!Þ ¼ Pð!ÞH1ð!Þ ð7:62Þ The problem therefore reduces to the white noise case in Theorem 13. The filter H2ð!Þ has to be matched to the output of the filter H1ð!Þ and thus reads H2ð!Þ ¼ PÃ1ð!Þ SN1N1 ð!Þ expðÀj!t0Þ ¼ PÃð!ÞH1Ãð!Þ expðÀj!t0Þ ð7:63Þ In the second equation above we used the fact that SN1N1 ð!Þ ¼ 1, which follows from Equations (7.61) and (4.27). The matched filter for a known signal pðtÞ disturbed by coloured noise is found when using Equations (7.61) and (7.63): Hð!Þ ¼ H1ð!ÞH2ð!Þ ¼ PÃð!Þ SNN ð!Þ expðÀj!t0Þ ð7:64Þ It is concluded that the matched filter for a signal disturbed by coloured noise corresponds to the optimal filter from Equation (7.54). Example 7.4: Consider the signal & xðtÞ ¼ at; 0 < t T 0; elsewhere ð7:65Þ This signal is shown in Figure 7.7(a). We want to characterize the matched filter for this signal when it is disturbed by white noise and to determine the maximum value of the 170 DETECTION AND OPTIMAL FILTERING aT x(t) (a) T x(−t) t (b) −T t hopt(t)=x(2T−t) (c) x(t1−τ) 2T hopt(τ) t (d) hopt(τ) t1 x(t2−τ) τ (e) t2 y(t) τ (f) 0 T 2T 3T t Figure 7.7 The different signals belonging to the example on a matched filter for the signal xðtÞ disturbed by white noise signal-to-noise ratio. The sampling instant is chosen as t0 ¼ 2T. In view of the simple description of the signal in the time domain it seems reasonable to do all the necessary calculations in the time domain. Illustrations of the different signals involved give a clear insight of the method. The signal xðÀtÞ is in Figure 7.7(b) and from this follows the optimal filter characterized by its impulse response hoptðtÞ, which is depicted in Figure 7.7(c). The maximum signal-to-noise ratio, occurring at the sampling instant t0, is calculated as follows. The noise power follows from Equation (7.58) yielding PN0 ¼ Z N0 T 20 a2t2 dt ¼ N0 a2 2 1 3 t3 T 0 ¼ N0 a2 T 3 6 ð7:66Þ THE CORRELATION RECEIVER 171 The signal value at t ¼ t0, using Equation (7.57), is yðt0Þ ¼ ZT 0 a2t2 dt ¼ a2 T 3 3 ð7:67Þ Using Equations (7.47), (7.66) and (7.67) the signal-to-noise ratio at the sampling instant t0 is S ¼ y2ðt0Þ ¼ a4T6=9 ¼ 2a2T3 N PN0 N0a2T3=6 3N0 ð7:68Þ The output signal yðtÞ follows from the convolution of xðtÞ and hoptðtÞ as given by Equation (7.56). The convolution is Z1 yðtÞ ¼ hoptð Þxðt À  Þ d ð7:69Þ À1 The various signals are shown in Figure 7.7. In Figure 7.7(d) the function hoptðÞ has been drawn, together with xðt À Þ for t ¼ t1; the latter is shown as a dashed line. We distinguish two different situations, namely t < t0 ¼ 2T and t > t0 ¼ 2T. In Figure 7.7(e) the latter case has been depicted for t ¼ t2. These pictures reveal that yðtÞ has an even symmetry with respect to t0 ¼ 2T. That is why we confine ourselves to calculate yðtÞ for t 2T. Moreover, from the figures it is evident that yðtÞ equals zero for t < T and t > 3T. For T t 2T we obtain (see Figure 7.7(d)) Zt yðtÞ ¼ aðÀ þ 2TÞaðÀ þ tÞ d T Zt ¼ a2 ð2 À t À 2T þ 2TtÞ d T ¼ a2 1  3 À t þ 2T 2 þ !t 2Tt 3 2 T! ¼ a2 À 1 t3 þ Tt2 À 3 T2t þ 2 T3 ; T t 2T 6 2 3 ð7:70Þ The function yðtÞ has been depicted in Figure 7.7(f). It is observed that the signal attains it maximum at t ¼ 2T, the sampling instant. & The maximum of the output signal of a matched filter is always attained at t0 and yðtÞ always shows even symmetry with respect to t ¼ t0. 7.3 THE CORRELATION RECEIVER In the former section we derived the linear time-invariant filter that maximizes the signal-tonoise ratio; it was called a matched filter. It can be used as a receiver filter prior to detection. It was shown in Section 7.1.2 that sampling and comparing the filtered signal with the proper threshold provides optimum detection of data signals in Gaussian noise. Besides matched filtering there is yet another method used to optimize the signal-to-noise ratio and which serves as an alternative for the matched filter. The method is called correlation reception. 172 DETECTION AND OPTIMAL FILTERING x(t )+N(t ) T (.) dt 0 x(t ) Figure 7.8 Scheme of the correlation receiver The scheme of the correlation receiver is presented in Figure 7.8. In the receiver a synchronized replica of the information signal xðtÞ has to be produced; this means that the signal must be known by the receiver. The incoming signal plus noise is multiplied by the locally generated xðtÞ and the product is integrated. In the sequel we will show that the output of this system has the same signal-to-noise ratio as the matched filter. For the derivation we assume that the pulse xðtÞ extends from t ¼ 0 to t ¼ T. Moreover, the noise process NðtÞ is supposed to be white with spectral density N0=2. Since the integration is a linear operation, it is allowed to consider the two terms of the product separately. Applying only xðtÞ to the input of the system of Figure 7.8 yields, at the output and at the sampling moment t0 ¼ T, the quantity ZT yðt0Þ ¼ x2ðtÞ dt ¼ Ex 0 ð7:71Þ where Ex is the energy in the pulse xðtÞ. Next we calculate the power of the output noise as ZT ZT ! PN0 ¼ E½N02ðtފ ¼ E NðtÞ xðtÞ dt Nð Þ xðÞ d 2Z0ZT 0 3 ¼ E4 NðtÞNðÞ xðtÞxðÞ dt d5 0 ZZT ¼ E½NðtÞNðފ xðtÞxðÞ dt d 0 ZZT ¼ N0 ðt À ÞxðtÞxðÞ dt d 2 ¼ 0 N0 2 ZT 0 x2ðtÞ dt ¼ N0 2 Ex ð7:72Þ Then the signal-to-noise ratio is found from Equations (7.71) and (7.72) as S N ¼ j yðt0Þj PN0 2 ¼ Ex2 N0 2 Ex ¼ 2Ex N0 ð7:73Þ This is exactly the same as Equation (7.60). THE CORRELATION RECEIVER 173 From the point of view of S=N, the matched filter receiver and the correlation receiver behave identically. However, for practical application it is of importance to keep in mind that there are crucial differences. The correlation receiver needs a synchronized replica of the known signal. If such a replica cannot be produced or if it is not exactly synchronized, the calculated signal-to-noise ratio will not be achieved, yielding a lower value. Synchronization is the main problem in using the correlation receiver. In many carrier-modulated systems it is nevertheless employed, since in such situations the phased–locked loop provides an excellent expedient for synchronization. The big advantage of the correlation receiver is the fact that all the time it produces, apart from the noise, the squared value of the signal. Together with the integrator this gives a continuously increasing value of the output signal, which makes the receiver quite invulnerable to deviations from the optimum sampling instant. This is in contrast to the matched filter receiver. If, for instance, the information signal changes its sign, as is the case in modulated signals, then the matched filter output changes as well. In this case a deviation from the optimum sampling instant can result in the wrong decision about the information bit. This is clearly demonstrated by the next example. Example 7.5: In the foregoing it was shown that the matched filter and the correlation receiver have equal per- formance as far as the signal-to-noise ratios at the sampling instant is concerned. In this example we compare the outputs of the two receivers when an ASK modulated data signal has to be received. It suffices to consider a single data pulse isolated in time. Such a signal is written as & xðtÞ ¼ A cosð!0tÞ; 0; 0 t T the response depends on the specific design of the receiver. In an integrate-and-dump receiver the signal is sampled and subsequently the value of the integrator output is reset to zero. The response of Equation (7.77) is presented in Figure 7.10 for the same parameters as used to produce Figure 7.9. Note that in this case the response is continuously non-decreasing, so no change of sign occurs. This makes this type of receiver much less vulnerable to timing jitter of the sampler. However, a perfect synchronization is required instead. & 7.4 FILTERS THAT MINIMIZE THE MEAN-SQUARED ERROR Thus far it was assumed that the signal to be detected had a known shape. Now we proceed with signals that are not known in advance, but the shape of the signal itself has to be estimated. Moreover, we assume as in the former case that the signal is corrupted by additive noise. Although the signal is not known in the deterministic sense, some assumptions will be made about its stochastic properties; the same holds for the noise. In this section we make an estimate of the received signal in the mean-squared sense, i.e. we minimize the meansquared error between an estimate of the signal based on available data consisting of signal plus noise and the actual signal itself. As far as the signal processing is concerned we confine the treatment to linear filtering. Two different problems are considered. 1. In the first problem we assume that the data about the signal and noise are available for all times, so causality is ignored. We look for a linear time-invariant filtering that produces an optimum estimate for all times of the signal that is disturbed by the noise. This optimum linear filtering is called smoothing. 2. In the second approach causality is taken into account. We make an optimum estimate of future values of the signal based on observations in the past up until the present time. Once more the estimate uses linear time-invariant filtering and we call the filtering prediction. 7.4.1 The Wiener Filter Problem Based on the description in the foregoing we consider a realization SðtÞ of a wide-sense stationary process, called the signal. The signal is corrupted by the realization NðtÞ of another wide-sense stationary process, called the noise. Furthermore, the signal and noise are supposed to be jointly wide-sense stationary. The noise is supposed to be added to the signal. To the input of the estimator the process XðtÞ ¼ SðtÞ þ NðtÞ ð7:78Þ is applied. When estimating the signal we base the estimate ^Sðt þ TÞ at some time t þ T on a linear filtering of the input data XðtÞ, i.e. Zb ^Sðt þ TÞ ¼ hðÞXðt À Þ d ð7:79Þ a 176 DETECTION AND OPTIMAL FILTERING where hðÞ is the weighting function (equal to the impulse response of the linear timeinvariant filter) and the integration limits a and b are to be determined later. Using Equation (7.79) the mean-squared error is defined as e ¼4  E fSðt þ T Þ À ^Sðt þ T Þg2à ¼ "& Z E Sðt þ TÞ À b hðÞXðt À Þ '2# d a ð7:80Þ Now the problem is to find the function hðÞ that minimizes the functional expression of Equation (7.80). In the minimization process the time shift T and integration interval are fixed; later on we will introduce certain restrictions to the shift and the integration interval, but for the time being they are arbitrary. The minimization problem can be solved by applying the calculus of variations [16]. According to this approach extreme values of the functional are achieved when the function hðÞ is replaced by hðÞ þ gðÞ, where gðÞ is an arbitrary function of the same class as hðÞ. Next the functional is differentiated with respect to  and the result equated to zero for  ¼ 0. Solving the resulting equation produces the function hðÞ, which leads to the extreme value of the functional. In the next subsections we will apply this procedure to the problem at hand. 7.4.2 Smoothing In the smoothing (or filtering) problem it is assumed that the data (or observation) XðtÞ are known for the entire time axis À1 < t < 1. This means that there are no restrictions on the integration interval and we take a ! À1 and b ! 1. Expanding Equation (7.80) yields "Z 1 2# e ¼ E½S2ðt þ Tފ þ E hðÞXðt À Þ d À1 Z1 ! À E 2Sðt þ TÞ hðÞXðt À Þ dÞ À1 ð7:81Þ Evaluating the expectations we obtain Z1Z e ¼ RSSð0Þ þ hðÞhðÞE½Xðt À ÞXðt À ފ d d Z 1 À1 À 2 hðÞE½Sðt þ TÞXðt À ފ d À1 ð7:82Þ and further Z1Z Z1 e ¼ RSSð0Þ þ hðÞhðÞRXXð À Þ d d À 2 hðÞRSXðÀ À TÞ d À1 À1 ð7:83Þ FILTERS THAT MINIMIZE THE MEAN-SQUARED ERROR 177 According to the calculus of variations we replace hðÞ by hðÞ þ gðÞ and obtain Z1Z e ¼ RSSð0Þ þ ½hðÞ þ gðފ½hðÞ þ gðފRXXð À Þ d d Z 1 À1 À 2 ½hðÞ þ gðފRSXðÀ À TÞ d À1 ð7:84Þ The procedure proceeds by setting dde¼0¼ 0 ð7:85Þ After some straightforward calculations this leads to the solution Z1 RSXðÀ À TÞ ¼ RXSð þ TÞ ¼ hðÞRXXð À Þ d; À1 <  < 1 À1 ð7:86Þ Since we assumed that the data are available over the entire time axis we can imagine that we apply this procedure on stored data. Moreover, in this case the integral in Equation (7.86) can be Fourier transformed as SXSð!Þ expðj!TÞ ¼ Hð!Þ SXXð!Þ ð7:87Þ Hence we do not need to deal with the integral equation, which is now transformed into an algebraic equation. For the filtering problem we can set T ¼ 0 and the optimum filter follows immediately: Hoptð!Þ ¼ SXSð!Þ SXX ð!Þ ð7:88Þ In the special case that the processes SðtÞ and NðtÞ are independent and at least one of these processes has zero mean, then the spectra can be written as SXXð!Þ ¼ SSSð!Þ þ SNN ð!Þ SXSð!Þ ¼ SSSð!Þ ð7:89Þ ð7:90Þ and as a consequence the optimum filter characteristic becomes Hoptð!Þ ¼ SSSð!Þ SSSð!Þ þ SNN ð!Þ ð7:91Þ Once we have the expression for the optimum filter the mean-squared error of the estimate can be calculated. For this purpose multiply both sides of Equation (7.86) by hoptðÞ and integrate over . This reveals that, apart from the minus sign, the second term of Equation (7.83) is half of the value of the third term, so that Z1 emin ¼ RSSð0Þ À hoptð ÞRSXðÀ Þ d À1 ð7:92Þ 178 DETECTION AND OPTIMAL FILTERING If we define Z1 ðtÞ ¼4 RSSðtÞ À hoptð ÞRSXðt À Þ d À1 ð7:93Þ it is easy to see that emin ¼ ð0Þ The Fourier transform of ðtÞ is SSSð!Þ À Hoptð!Þ SSX ð!Þ ¼ SSSð!Þ À SSSð!Þ SXX SSX ð!Þ ð!Þ ð7:94Þ ð7:95Þ Hence the minimum mean-squared error is emin ¼ 1 2p Z1 À1 ! SSS ð!Þ À SSS ð!Þ SXX SSX ð!Þ ð!Þ d! ð7:96Þ When the special case of independence of the processes SðtÞ and NðtÞ is once more invoked, i.e. Equations (7.89) and (7.90) are inserted, then emin ¼ 1 2p Z1 À1 SSSð!Þ SNN ð!Þ SSSð!Þ þ SNN ð!Þ d! ð7:97Þ Example 7.6: A wide-sense stationary process has a flat spectrum within a limited frequency band, i.e. & SSSð!Þ ¼ S=2; j!j W 0; j!j > W ð7:98Þ The noise is independent of SðtÞ and has a white spectrum with a spectral density of N0=2. In this case the optimum smoothing filter has the transfer function 8 Hoptð!Þ ¼ < : S S þ N0 0; ; j!j W j!j > W ð7:99Þ This result can intuitively be understood; namely the signal spectrum is completely passed undistorted by the ideal lowpass filter of bandwidth W and the noise is removed outside the signal bandwidth. The estimation error is emin ¼ 1 2p ZW 0 S N0 S þ N0 d! ¼ W 2p S N0 S þ N0 ¼ W 2p S S=N0 þ 1 ð7:100Þ Interpreting S=N0 as the signal-to-noise ratio it is observed that the error decreases with increasing signal-to-noise ratios. For a large signal-to-noise ratio the error equals the noise power that is passed by the filter. & FILTERS THAT MINIMIZE THE MEAN-SQUARED ERROR 179 The filtering of an observed signal as described by Equation (7.91) is also called signal restoration. It is the most obvious method when the observation is available as stored data. When this is not the case, but an incoming signal has to be processed in real time, then the filtering given in Equation (7.88) can be applied, provided that the delay is so large that virtually the whole filter response extends over the interval À1 < t < 1. Despite the realtime processing, a delay between the arrival of the signal SðtÞ and the estimate of Sðt À TÞ should be allowed. In that situation the optimum filter has an extra factor of expðÀj!TÞ, which provides the delay, as follows from Equation (7.87). In general, a longer delay will reduce the estimation error, as long as the delay is shorter than the duration of the filter’s impulse response hðÞ. 7.4.3 Prediction We now consider prediction based on the observation up to time t. Referring to Equation (7.79), we consider ^Sðt þ TÞ for positive values of T whereas XðtÞ is only known up to t. Therefore the integral limits in Equation (7.79) are a ¼ À1 and b ¼ t. We introduce the causality of the filter’s impulse response, given as hðtÞ ¼ 0; for t < 0 ð7:101Þ The general prediction problem is quite complicated [4]. Therefore we will confine the considerations here to the simplified case where the signal SðtÞ is not disturbed by noise, i.e. now we take NðtÞ  0. This is called pure prediction. It is easy to verify that in this case Equation (7.86) is reduced to Z1 RSSð þ TÞ ¼ hðÞRSSð À Þ d;  ! 0 0 ð7:102Þ This equation is known as the Wiener–Hopf integral equation. The solution is not as simple as in former cases. This is due to the fact that Equation (7.102) is only valid for  ! 0; therefore we cannot use the Fourier transform to solve it. The restriction  ! 0 follows from Equation (7.84). In the case at hand the impulse response hðÞ of the filter is supposed to be causal and the auxiliary function gðÞ should be of the same class. Consequently, the solution now is only valid for  ! 0. For  < 0 the solution should be zero, and this should be guaranteed by the solution method. Two approaches are possible for a solution. Firstly, a numerical solution can be invoked. For a fixed value of T the left-hand part of Equation (7.102) is sampled and the samples are collected in a vector. For the right-hand side RSSð À Þ is sampled for each value of . The different vectors, one for each , are collected in a matrix, which is multiplied by the unknown vector made up from the sampled values of hðÞ. Finally, the solution is produced by matrix inversion. Using an approximation we will also be able to solve it by means of the Laplace transform. Each function can arbitrarily be approximated by a rational function, the fraction of two polynomials. Let us suppose that the bilateral Laplace transform [7] of the autocorrelation function of SðtÞ is written as a rational function, i.e. SSSðpÞ ¼ Aðp2Þ Bðp2Þ ð7:103Þ 180 DETECTION AND OPTIMAL FILTERING Since the spectrum is an even function it can be written as a function of p2. If we look at the positioning of zeros and poles in the complex p plane, it is revealed that this pattern is symmetrical with respect to the imaginary axis; i.e. if pi is a root of Aðp2Þ then Àpi is a root as well. The same holds for Bðp2Þ. Therefore SSSðpÞ can be factored as SSSðpÞ ¼ CðpÞ DðpÞ CðÀpÞ DðÀpÞ ¼ KðpÞ KðÀpÞ ð7:104Þ where CðpÞ and DðpÞ comprise all the roots in the left half-plane and CðÀpÞ and DðÀpÞ the roots in the right half-plane, respectively; CðpÞ and CðÀpÞ contain the roots of A2ðpÞ and DðpÞ and DðÀpÞ those of B2ðpÞ. For the sake of convenient treatment we suppose that all roots are simple. Moreover, we define KðpÞ ¼4 CðpÞ DðpÞ ð7:105Þ Both this function and its inverse are causal and realizable, since they are stable [7]. Let us now return to Equation (7.102), the integral equation to be solved. Rewrite it as Z1 RSSð þ TÞ ¼ hðÞRSSð À Þ d þ f ðÞ 0 ð7:106Þ where f ðÞ is a function that satisfies f ðÞ ¼ 0; for  ! 0 ð7:107Þ i.e. f ðÞ is anti-causal and analytic in the left-hand p plane ðRefpg < 0Þ. The Laplace transform of Equation (7.106) is SSSðpÞ expðpTÞ ¼ SSSðpÞ HðpÞ þ FðpÞ ð7:108Þ where HðpÞ is the Laplace transform of hðtÞ and FðpÞ that of f ðtÞ. Solving this equation for HðpÞ yields HðpÞ ¼4 NðpÞ MðpÞ ¼ expðpT Þ CðpÞ CðÀpÞ À FðpÞ CðpÞ CðÀpÞ DðpÞ DðÀpÞ ð7:109Þ with the use of Equation (7.104). This function may only have roots in the left half-plane. If we select FðpÞ ¼ CðÀpÞ DðÀpÞ ð7:110Þ then Equation (7.109) becomes HðpÞ ¼ NðpÞ MðpÞ ¼ expðpTÞ CðpÞ CðpÞ À DðpÞ ð7:111Þ FILTERS THAT MINIMIZE THE MEAN-SQUARED ERROR 181 The choice given by Equation (7.110) guarantees that f ðtÞ is anti-causal, i.e. f ðtÞ ¼ 0 for t ! 0. Moreover, making Mð pÞ ¼ Cð pÞ ð7:112Þ satisfies one condition on Hð pÞ, namely that it is an analytic function in the right half-plane. Based on the numerator of Equation (7.111) we have to select Nð pÞ; for that purpose the data of Dð pÞ can be used. We know that Dð pÞ has all its roots in the left half-plane, so if we select Nð pÞ such that its roots pi coincide with those of Dð pÞ then the solution satisfies the condition that Nð pÞ is an analytic function in the right half-plane. This is achieved when the roots pi are inserted in the numerator of Equation (7.111) to obtain expð piTÞ Cð piÞ ¼ Nð piÞ ð7:113Þ for all the roots pi of Dð pÞ. Connecting the roots of the solution in this way to the polynomial Dð pÞ guarantees on the one hand that Nð pÞ is analytic in the right half-plane and on the other hand satisfies Equation (7.111). This completes the selection of the optimum Hð pÞ. Summarizing the method, we have to take the following steps: 1. Factor the spectral function SSSðpÞ ¼ Cð Dð pÞ pÞ CðÀpÞ DðÀpÞ ð7:114Þ where Cð pÞ and Dð pÞ comprise all the roots in the left half-plane and CðÀpÞ and DðÀpÞ the roots in the right half-plane, respectively. 2. The denominator of the optimum filter Hð pÞ has to be taken equal to Cð pÞ. 3. Expand Kð pÞ into partial fractions: KðpÞ ¼ Cð Dð pÞ pÞ ¼ p a1 À p1 þ Á Á Á þ p an À pn ð7:115Þ where pi are the roots of Dð pÞ. 4. Construct the modified polynomial Kmð pÞ ¼ expð p1 T Þ p a1 À p1 þ Á Á Á þ expð pn T Þ p an À pn ð7:116Þ 5. The optimum filter, described in the Laplace domain, then reads Hoptð pÞ ¼ Kmð pÞ Dð Cð pÞ pÞ ¼ Nð Cð pÞ pÞ ð7:117Þ Example 7.7: Assume a process with the autocorrelation function RSSðÞ ¼ expðÀ j j Þ; > 0 ð7:118Þ 182 DETECTION AND OPTIMAL FILTERING Then from a Laplace transform table it follows that SSSð pÞ ¼ 2 2 À p2 ð7:119Þ which is factored into pffiffiffiffiffi pffiffiffiffiffi SSSð pÞ ¼ 2 þp 2 Àp ð7:120Þ For the intermediate polynomial KðpÞ it is found that pffiffiffiffiffi K ð pÞ ¼ 2 þp ð7:121Þ Its constituting polynomials are pffiffiffiffiffi Cð pÞ ¼ 2 ð7:122Þ and Dð pÞ ¼ þ p ð7:123Þ The polynomial Cð pÞ has no roots and the only root of Dð pÞ is p1 ¼ À . This produces pffiffiffiffiffiffi Kmð pÞ ¼ 2 þp expðÀ T Þ ð7:124Þ so that finally for the optimum filter we find Hoptð pÞ ¼ Nð Mð pÞ pÞ ¼ Kmð pÞ Dð Cð pÞ pÞ ¼ expðÀ T Þ ð7:125Þ and the corresponding impulse response is hoptðtÞ ¼ expðÀ TÞ ðtÞ ð7:126Þ The minimum mean-squared error is given by substituting zero for the lower limit in the integral of Equation (7.92). This yields the error Z1 emin ¼ RSSð0Þ À hð ÞRSSðÀ Þ d ¼ 1 À expðÀ TÞ 0 ð7:127Þ & This result reflects what may be expected, namely the facts that the error is zero when T ¼ 0, which is actually no prediction, and that the error increases with increasing values of T. FILTERS THAT MINIMIZE THE MEAN-SQUARED ERROR 183 7.4.4 Discrete-Time Wiener Filtering Discrete-Time Smoothing: Once the Wiener filter for continuous processes has been analysed, the time-discrete version follows straightforwardly. Equation (7.86) is the general solution for describing the different situations considered in this section. Its time-discrete version when setting the delay to zero is X 1 RXS½nŠ ¼ h½mŠ RXX½n À mŠ; for all n m¼À1 ð7:128Þ Since this equation is valid for all n it is easily solved by taking the z-transform of both sides: ~SXSðzÞ ¼ H~ ðzÞ ~SXXðzÞ ð7:129Þ or H~ opt ðzÞ ¼ ~SXSðzÞ ~SXX ðzÞ ð7:130Þ The error follows from the time-discrete counterparts of Equation (7.92) or Equation (7.96). If both RXX½nŠ and RXS½nŠ have finite extent, let us say RXX½nŠ ¼ RXS½nŠ ¼ 0 for j nj > N, and if the extent of h½mŠ is limited to the same range, then Equation (7.128) can directly be solved in the time domain using matrix notation. For this case we define the ð2N þ 1Þ Â ð2N þ 1Þ matrix as 2 RXX½0Š RXX½1Š RXX½2Š Á Á Á RXX½NŠ 0 ÁÁÁ RXX ¼4 66666666664 RXX ½1Š ... 0 ... RXX ½0Š ... 0 ... RXX ½1Š ... RXX ½N Š ... ÁÁÁ ... ÁÁÁ ... RXX½N À 1Š ... RXX ½0Š ... RXX ½N Š ... RXX ½1Š ... ÁÁÁ ... ÁÁÁ ... 03 0 ... 0 ... 77777777775 0 0 0 Á Á Á RXX½NŠ RXX½N À 1Š Á Á Á RXX½0Š ð7:131Þ Moreover, we define the ð2N þ 1Þ element vectors as RTXS ¼4  RXS½ÀNŠ RXS½ÀN þ 1Š Á Á Á RXS½0Š Á Á Á RXS½N À 1Š à RXS½NŠ ð7:132Þ and hT ¼4  h½ÀNŠ h½ÀN þ 1Š Á Á Á h½0Š Á Á Á h½N À 1Š à h½NŠ ð7:133Þ where RTXS and hT are the transposed vectors of the column vectors RXS and h, respectively. 184 DETECTION AND OPTIMAL FILTERING By means of these definitions Equation (7.128) is rewritten as RXS ¼ RXX Á h ð7:134Þ with the solution for the discrete-time Wiener smoothing filter hopt ¼ RÀXX1 Á RXS ð7:135Þ This matrix description fits well in a modern numerical mathematical software package such as Matlab, which provides compact and efficient programming of matrices. Programs developed in Matlab can also be downloaded into DSPs, which is even more convenient. Discrete-Time Prediction: For the prediction problem a discrete-time version of the method presented in Subsection 7.4.3 can be developed (see reference [12]). However, using the time domain approach presented in the former paragraph, it is very easy to include noise; i.e. there is no need to limit the treatment to pure prediction. Once more we start from the discrete-time version of Equation (7.86), which is now written as X 1 RXS½n þ KŠ ¼ h½mŠ RXX½n À mŠ; for all n m¼0 ð7:136Þ since the filter should be causal, i.e. h½mŠ ¼ 0 for m < 0. Comparing this equation with Equation (7.128) reveals that they are quite similar. There is a time shift in RXS and a difference in the range of h½mŠ. For the rest the equations are the same. This means that the solution is also the same, provided that the matrix RXX and the vectors RTXS and hT are accordingly redefined. They become the ð2N þ 1Þ Â ðN þ 1Þ matrix 2 RXX½NŠ RXX ¼4 666666664 RXX½N À ... RXX ½0Š ... 1Š 0 RXX ½N Š ... RXX ½1Š ... ÁÁÁ 0 3 ÁÁÁ ... ÁÁÁ ... 0 ... RXX ½N ... Š 777777775 RXX½NŠ RXX½N À 1Š Á Á Á RXX½0Š ð7:137Þ and the ð2N þ 1Þ element vector RTXS ¼4  RXS½ÀN þ KŠ RXS½ÀN þ 1 þ KŠ Á Á Á RXS½N À 1 þ KŠ RXS½N þ à KŠ ð7:138Þ respectively, and the ðN þ 1Þ element vector hT ¼4  h½0Š h½1Š Á Á Á h½N À 1Š à h½NŠ ð7:139Þ PROBLEMS 185 The estimation error follows from the discrete-time version of Equation (7.92), which is XN e ¼ RSS½0Š À h½nŠ RSX½ÀnŠ n¼0 ð7:140Þ When the noise N½nŠ has zero mean and S½nŠ and N½nŠ are independent, they are orthogonal. This simplifies the cross-correlation of RSX to RSS. 7.5 SUMMARY The optimal detection of binary signals disturbed by noise has been considered. The problem is reduced to hypothesis testing. When the noise has a Gaussian probability density function, we arrive at a special form of linear filtering, the so-called matched filtering. The optimum receiver for binary data signals disturbed by additive wide-sense stationary Gaussian noise consists of a matched filter followed by a sampler and a decision device. Moreover, the matched filter can also be applied in situations where the noise (not necessarily Gaussian) has to be suppressed maximally compared to the signal value at a specific moment in time, called the sampling instant. Since the matched filter is in fact a linear time-invariant filter and the input noise is supposed to be wide-sense stationary, this means that the output noise variance is constant, i.e. independent of time, and that the signal attains its maximum value at the sampling instant. The name matched filter is connected to the fact that the filter characteristic (let it be described in the time or in the frequency domain) is determined by (matched to) both the shape of the received signal and the power spectral density of the disturbing noise. Finally, filters that minimize the mean-squared estimation error (Wiener filters) have been derived. They can be used for smoothing of stored data or portions of a random signal that arrived in the past. In addition, filters that produce an optimal prediction of future signal values have been described. Such filters are derived both for continuous processes and discrete-time processes. 7.6 PROBLEMS 7.1 The input R ¼ P þ N is applied to a detector. The random variable P represents the information and is selected from P 2 fþ1; À0:5g and the selection occurs with the probabilities PðP ¼ þ1Þ ¼ 1 4 and PðP ¼ À0:5Þ ¼ 34. The noise N has a triangular distribution fNðnÞ ¼ triðnÞ. (a) Make a sketch of the weighted (by the prior probabilities) conditional distribution functions. (b) Determine the optimal decision regions. (c) Calculate the minimum error probability. 7.2 Consider a signal detector with input R ¼ P þ N. The random variable P is the information and is selected from P 2 fþA; ÀAg, with A a constant, and this selection 186 DETECTION AND OPTIMAL FILTERING occurs with equal probability. The noise N is characterized by the Laplacian probability density function  pffiffi  fN ðnÞ ¼ p1 ffiffi 2 exp À 2 j nj  (a) Determine the decision regions, without making a calculation. (b) Consider the minimum probability of error receiver. Derive the probability of error for this receiver as a function of the parameters A and . (c) Determine the variance of the noise. (d) Defining an appropriate signal-to-noise ratio S=N, determine the S=N to achieve an error probability of 10À5. 7.3 The M-ary PSK (phase shift keying) signal is defined as & ' pðtÞ ¼ A cos !0 t þ ði À 1Þ 2p M ; i ¼ 1; 2; . . . ; M; for 0 t T where A and !0 are constants representing the carrier amplitude and frequency, respectively, and i is randomly selected depending on the codeword to be transmitted. In Appendix A this signal is called a multiphase signal. This signal is disturbed by wide-sense stationary white Gaussian noise with spectral density N0=2. (a) Make a picture of the signal constellation in the signal space for M ¼ 8. (b) Determine the decision regions and indicate them in the picture. (c) Calculate the symbol error probability (i.e. the probability that a codeword is detected in error) for large values of the signal-to-noise ratio; assume, among others, that this error rate is dominated by transitions to nearest neighbours in the signal constellation. Express this error probability in terms of M, the mean energy in the codewords and the noise spectral density. Hint: use Equation (5.65) for the probability density function of the phase. 7.4 A filter matched to the signal xðtÞ ¼ 8 < :  A1 0; À j tj T  ; 0 < j tj < T elsewhere has to be realized. The signal is disturbed by noise with the power spectral density SNN ð!Þ ¼ W W2 þ !2 with A, T and W positive, real constants. (a) Determine the Fourier transform of xðtÞ. (b) Determine the transfer function Hoptð!Þ of the matched filter. (c) Calculate the impulse response hoptðtÞ. Sketch hoptðtÞ. PROBLEMS 187 (d) Is there any value of t0 for which the filter becomes causal? If so, what is that value? 7.5 The signal xðtÞ ¼ uðtÞ expðÀWtÞ, with W > 0 and real, is applied to a filter together with white noise that has a spectral density of N0=2. (a) Calculate the transfer function of the filter that is matched to xðtÞ. (b) Determine the impulse response of this filter. Make a sketch of it. (c) Is there a value of t0 for which the filter becomes causal? (d) Calculate the maximum signal-to-noise ratio at the output. 7.6 In a frequency domain description as given in Equation (7.54) the causality of the matched filter cannot be guaranteed. Using Equation (7.54) show that for a matched filter for a signal disturbed by (coloured) noise the following integral equation gives a time domain description: Z1 hoptðÞRNN ðt À Þ d ¼ xðt0 À tÞ À1 where the causality of the filter can now be guaranteed by setting the lower bound of the integral equal to zero. 7.7 A pulse & xðtÞ ¼ A cosðpt=TÞ; j tj T=2 0; j tj > T=2 is added to white noise NðtÞ with spectral density N0=2. Find ðS=NÞmax for a filter matched to xðtÞ and NðtÞ. 7.8 The signal xðtÞ is defined as & xðtÞ ¼ A; 0; 0 t wÞ ¼ Pð0; wÞ ¼ expðÀwÞ; w ! 0 ð8:21Þ and & PðW wÞ ¼ 1 À expðÀwÞ; 0; w!0 w<0 ð8:22Þ Thus, the waiting time is an exponential random variable and its probability density function is written as & fW ðwÞ ¼ dfPðW dw wÞg ¼  expðÀwÞ; w ! 0 0; w<0 ð8:23Þ 198 POISSON PROCESSES AND SHOT NOISE The mean waiting time has the value E½W Š ¼ Z 0 1 w expðÀwÞ dw ¼ 1  ð8:24Þ This result can also easily be understood when remembering that  is the mean number of events per unit of time. Then the mean waiting time will be its inverse. 8.3 THE HOMOGENEOUS POISSON PROCESS Let us consider an homogeneous Poisson process defined as the sum of impulses X XðtÞ ¼ ðt À tiÞ i ð8:25Þ where the  impulses ðt À tiÞ appear at random times ftig governed by a Poisson distribution. This process can be used for modelling such physical processes as the detection of photons by a photodetector. Each time an arriving photon is detected, it causes a small impulse-shaped amount of current having an amplitude equal to the charge of an electron. This means that the photon arrival and thus the production of current impulses may be described by Equation (8.25), depicted as the input of Figure 8.1(b). Due to the travel time of the moving charges in the detector and the frequency-dependent load circuit (see Figure 8.1(a)), the response of the current through the load will have a different shape, for instance as given in Figure 8.1(b). The shape of this response is called hðtÞ, being X(t ) Y(t ) (a) t h(t ) t linear time-invariant system (b) t Figure 8.1 (a) Photodetector circuit with load and (b) corresponding Poisson process and shot noise process THE HOMOGENEOUS POISSON PROCESS 199 the impulse response of the circuit. The total voltage across the load is described by the process X YðtÞ ¼ hðt À tiÞ i ð8:26Þ This process is called shot noise and even if the rate  of the process is constant (in the example of the photodetector the constant amount of optical power arriving), the process YðtÞ will indeed show a noisy character, as has been depicted in the output of Figure 8.1(b). The upper curve at the output represents the individual responses, while the lower curve shows the sum of these pulses. Calculating the probability density function of this process is a difficult problem, but we will be able to calculate its mean value and autocorrelation function, and thus its spectrum as well. 8.3.1 Filtering of Homogeneous Poisson Processes and Shot Noise In order to simplify the calculation of the mean value and the autocorrelation function of the shot noise, the time axis is subdivided into a sequence of consecutive small time intervals of equal width Át. These time intervals are taken so short that  Át ( 1. Next we define the random variable Vn such that for all integer values n & Vn ¼ 0; if no impulse occurs in the interval n Át < t < ðn þ 1Þ Át 1; if one impulse occurs in the interval n Át < t < ðn þ 1Þ Át ð8:27Þ In the sequel we shall neglect the probability that more than one impulse will occur in such a brief time interval as Át. Following the definition of a Poisson process as given in the introduction to this chapter, it is concluded that the random variables Vn and Vm are independent if n 6¼ m. The probabilities of the two possible realizations of Vn read PðVn ¼ 0Þ ¼ expðÀ ÁtÞ % 1 À  Át PðVn ¼ 1Þ ¼  Át expðÀ ÁtÞ %  Át ð8:28Þ ð8:29Þ where the exact expressions are achieved by inserting k ¼ 0 and k ¼ 1, respectively, in Equation (8.1) and the approximations follow from the fact that  Át ( 1. The expectation of the random variable Vn is E½VnŠ ¼ 0  PðVn ¼ 0Þ þ 1  PðVn ¼ 1Þ ¼ PðVn ¼ 1Þ ¼  Át ð8:30Þ Based on the foregoing it is easily revealed that the process YðtÞ is approximated by the process X 1 Y^ðtÞ ¼ Vn hðt À n ÁtÞ n¼À1 ð8:31Þ 200 POISSON PROCESSES AND SHOT NOISE The smaller the Át the closer the approximation Y^ðtÞ will approach the shot noise process YðtÞ; in the limit of Át approaching zero, the processes will merge. The expectation of Y^ðtÞ is X 1 X 1 E½Y^ðtފ ¼ E½VnŠ hðt À n ÁtÞ ¼  Át hðt À n ÁtÞ n¼À1 n¼À1 ð8:32Þ When Át is made infinitesimally small then the summation is converted into an integral and the expected value of the shot noise process is obtained: Z1 Z1 E½Yðtފ ¼  hðt À Þ d ¼  hðÞ d À1 À1 ð8:33Þ In order to gain more information about the shot noise process we will consider its characteristic function. This function is deduced in a straightforward way using the approximation Y^ðtÞ: ÈY^ ðuÞ ¼ E½expfjuY^ðtÞgŠ "( )# X 1 ¼ E exp ju Vn hðt À n ÁtÞ n¼À1 Y1 ¼ E½expfjuVn hðt À n ÁtÞgŠ n¼À1 ð8:34Þ In Equation (8.34) the change from the summation in the exponential to the product of the expectation of the exponentials is allowed since the random variables Vn are independent. Invoking the law of total probability, the characteristic function of Y^ðtÞ is written as Y1 ÈY^ ðuÞ ¼ PðVn ¼ 0Þ Â 1 þ PðVn ¼ 1Þ exp½juhðt À n Átފ n¼À1 ð8:35Þ and using Equations (8.28) and (8.29) gives Y1 ÈY^ ðuÞ ¼ f1 À  Át þ  Át exp½juhðt À n Átފg n¼À1 Y1 ¼ ð1 þ  Átfexp½juhðt À n Átފ À 1gÞ n¼À1 ð8:36Þ Now we use the approximation 1 þ x % expðxÞ to proceed as Y1 ÈY^ ðuÞ % expð Átfexp½juhðt À n Átފ À 1gÞ n¼À1 ! X 1 ¼ exp  Átfexp½juhðt À n Átފ À 1g n¼À1 ð8:37Þ THE HOMOGENEOUS POISSON PROCESS 201 Once again Át is made infinitesimally small so that the summation converts into an integral and we arrive at the characteristic function of the shot noise process YðtÞ:  Z1  ÈY ðuÞ ¼ exp  fexp½juhðt À ފ À 1g d  ZÀ11  ¼ exp  fexp½juhðފ À 1g d ð8:38Þ À1 From this result several features of the shot noise can be deduced. By inverse Fourier transforming it, we find the probability density function of YðtÞ, but this is in general a difficult task. However, the mean and variance of the shot noise process follow immediately from the second characteristic function Z1 ÉY ðuÞ ¼ ln ÈY ðuÞ ¼  fexp½juhðފ À 1g d Z ¼ À1 1  juhð Þ À 1 2 u2h2 ð Þ þ Á Á à Á d À1 ð8:39Þ and by invoking Equation (8.19). & Theorem 15 The homogeneous shot noise process has the mean value Z1 E½Yðtފ ¼  hðÞ d À1 and variance Z1 2Y ¼  h2ðÞ d À1 These two equations together are named Campbell’s theorem. ð8:40Þ ð8:41Þ Actually, the result of Equation (8.40) was earlier derived in Equation (8.33), but here we found this result in an alternative way. We emphasize that both the mean value and variance are proportional to the Poisson parameter . When the mean value of the shot noise process is interpreted as the signal and the variance as the noise, then it is concluded that this type of noise is not additive, as in the classical communication model, but multiplicative; i.e. the noise variance is proportional to the signal value and the signal-to-shot noise ratio is proportional to . Example 8.4: When we take a rectangular pulse for the impulse response of the linear time-invariant filter in Figure 8.1, according to & hðtÞ ¼ 1; 0 t T 0; t < 0 and t > T ð8:42Þ 202 POISSON PROCESSES AND SHOT NOISE then it is found that ÈY ðuÞ ¼ expfT½expðjuÞ À 1Šg ð8:43Þ Comparing this result with Equation (8.9), it is concluded that in this case the output probability density function is a discrete one and gives the Poisson distribution of Equation (8.1). Actually, the filter gives as the output value at time ts the number of Poisson impulses that arrived in the past T seconds, i.e. in the interval t 2 fts À T; tsg. & Next we want to calculate the autocorrelation function RY ðt1; t2Þ of the shot noise process and from that the power spectrum. For that purpose we define the joint characteristic function of two random variables X1 and X2 as Z1Z Èðu1; u2Þ ¼4 E½expðju1X1 þ ju2X2ފ ¼ fXðx1; x2Þ expðju1x1 þ ju2x2Þ dx1 dx2 À1 ð8:44Þ Actually, this function is the two-dimensional Fourier transform of the joint probability density function of X1 and X2. In order to evaluate this function for the shot noise process YðtÞ we follow a similar procedure as before, i.e. we start with the approximating process Y^ðtÞ: ÈY^ ðu1; u2Þ ¼ E½expfju1Y^ðt1Þ þ ju2Y^ðt2ÞgŠ ð8:45Þ Elaborating this in a straightforward manner as before (see Equations (8.34) to (8.38)) we arrive at the joint characteristic function of YðtÞ:  Z1  ÈY ðu1; u2Þ ¼ exp  fexp½ju1hðt1 À Þ þ ju2hðt2 À ފ À 1g d À1 ð8:46Þ The second joint characteristic function reads Z1 ÉY ðu1; u2Þ ¼4 ln½ÈY ðu1; u2ފ ¼  fexp½ju1hðt1 À  Þ þ ju2hðt2 À ފ À 1g d À1 ð8:47Þ In the sequel we shall show that by series expansion of this latter function, the autocorrelation function can be calculated. However, we will first show how moments of several orders are generated by the joint characteristic function. For that purpose we apply series expansion to both exponentials in Equation (8.44): Z1Z Èðu1; u2Þ ¼ À1 fX ðx1 ; x2 Þ X 1 n¼0 ðju1x1Þn n! X 1 m¼0 ðju2x2Þm m! dx1 dx2 ¼ Z1Z fX ðx1 ;  x2Þ 1 þ ju1x1 À u21x21 2 þ Á Á  Á1 þ ju2x2 À u22x22 2 þ Á Á  Á dx1 dx2 À1 ¼ 1 þ ju1E½X1Š þ ju2E½X2Š À u1u2E½X1X2Š þ Á Á Á ð8:48Þ From this equation it is observed that the term with ju1 comprises E½X1Š, the term with ju2 comprises E½X2Š, the term with Àu1u2 comprises E½X1X2Š, etc. In fact, series expansion of THE HOMOGENEOUS POISSON PROCESS 203 the joint characteristic function generates all arbitrary moments as follows: Èðu1; u2Þ ¼ X 1 n¼0 X 1 m¼0 E½X1nX2mŠ ðju1Þnðju2Þm n!m! ð8:49Þ Since the characteristic function given by Equation (8.46) contains a double exponential, producing a series expansion is intractable. Therefore we make a series expansion of the second joint characteristic function of Equation (8.47) and identify from that expansion the second-order moment E½Yðt1ÞYðt2ފ we are looking for. This expansion is once again based on the series expansion of the logarithm lnð1 þ xÞ ¼ x À x2=2 þ Á Á Á: Éðu1; u2Þ ¼ ju1E½X1Š þ ju2E½X2Š À u1u2E½X1X2Š À 1 2 u21E½X12Š À 1 2 u22E½X22Š þ Á Á Á À 1 2 ðju1E½X1Š þ ju2E½X2Š À u1u2E½X1X2Š þ Á Á ÁÞ2 þ Á Á Á ¼ ju1E½X1Š þ ju2E½X2Š À u1u2E½X1X2Š À u21E½X12Š À u22E½X22Š þ Á Á Á À 12ðÀu21E2½X1Š À u22E2½X2Š À 2u1u2E½X1ŠE½X2Š þ Á Á ÁÞ þ Á Á Á ð8:50Þ When looking at the term with u1u2, we discover that its coefficient reads ÀðE½X1X2ŠÀ E½X1ŠE½X2ŠÞ. Comparing this expression with Equation (2.65) and applying it to the process YðtÞ it is revealed that this coefficient equals the negative of the autocovariance function CYY ðt1; t2Þ. As we have already calculated the mean value of the process (see Equations (8.33) and (8.40)), we can easily obtain its autocorrelation function. To evaluate this function we expand Equation (8.47) in a similar way and look for the coefficient of the term with u1u2. This yields Z1 Z1 CYY ðt1; t2Þ ¼  hðt1 À Þ hðt2 À Þ d ¼  hðÞ hðt2 À t1 þ Þ d À1 À1 ð8:51Þ We observe that this expression does not depend on the absolute time, but only on the difference t2 À t1. Since the mean was independent of time as well, it is concluded that the shot noise process is wide-sense stationary. Its autocorrelation function reads Z1 Z1 !2 RYY ðÞ ¼ CYY ðÞ þ E2½Yðtފ ¼  hðÞ hð þ Þ d þ 2 hðÞ d À1 À1 ð8:52Þ From this we can immediately find the power spectrum by Fourier transforming the latter expression: SYY ð!Þ ¼ jHð!Þj2 þ 2p2H2ð0Þð!Þ ð8:53Þ It can be seen that the spectrum always comprises a d.c. component if this component is passed by the filter, i.e. if Hð0Þ 6¼ 0. As a special case we consider the spectrum of the input process XðtÞ that consists of a sequence of Poisson impulses as given by Equation (8.25). This spectrum is easily found by inserting Hð!Þ ¼ 1 in Equation (8.53). Then, apart from the  function, the spectrum comprises a constant value, so this part of the spectrum behaves as white noise. 204 POISSON PROCESSES AND SHOT NOISE For large values of the Poisson parameter , the shot noise process approaches a Gaussian process [1]. This model is widely used for the shot noise generated by electronic components. 8.4 INHOMOGENEOUS POISSON PROCESSES An inhomogeneous Poisson process is a Poisson process for which the parameter  varies with time, i.e.  ¼ ðtÞ. In order to derive the important properties of such a process we redo the calculations of the preceding section, where the parameter  in Equations (8.28) to (8.37) is replaced by ðn ÁtÞ. The characteristic function then becomes Z 1  ÈY ðuÞ ¼ exp ðÞfexp½juhðt À ފ À 1g d À1 ð8:54Þ Based on Equation (8.19), the mean and variance of this process follows immediately: Z1 E½Yðtފ ¼ ðÞhðt À Þ d À1 Z1  2 Y ¼ ðÞh2ðt À Þ d À1 ð8:55Þ ð8:56Þ Actually, these two equations are an extension of Campbell’s theorem. Without going into detail, the autocorrelation function of this process is easily found in a way similar to the procedure of calculating the second joint characteristic function in the preceding section. The second joint characteristic function of the inhomogeneous Poisson process is obtained as Z1 ÉY ðu1; u2Þ ¼ ðÞfexp½ju1hðt1 À Þ þ ju2hðt2 À ފ À 1g d À1 ð8:57Þ and from this, once again, similarly to the preceding section, it follows that Z1 RYY ðt1; t2Þ ¼ ðÞhðt1 À Þ hðt2 À Þ d ÀZ1 1 Z1 þ ðÞhðt1 À Þ d ðÞhðt2 À Þ d À1 À1 ð8:58Þ Let us now suppose that the function ðtÞ is a stochastic process as well, which is independent of the Poisson process. Then the process YðtÞ is a doubly stochastic process [17]. Moreover, we assume that ðtÞ is a wide-sense stationary process. Then from Equation (8.55) it follows that the mean value of YðtÞ is independent of time. In the autocorrelation function we substitute t1 ¼ t and t2 ¼ t þ ; this yields Z1 RYY ðt; t þ Þ ¼ E½Š hðt À Þhðt þ  À Þ d À1 Z1Z þ Rð1 À 2Þhðt À 1Þhðt þ  À 2Þ d1 d2 ð8:59Þ À1 THE RANDOM-PULSE PROCESS 205 In this equation RðÁÞ is the autocorrelation function of the process ðtÞ. Further elaborating the Equation (8.59) gives Z1 RYY ðt; t þ Þ ¼ E½Š hðÞhð þ Þ d À1 Z1Z þ Rð þ 1 À 2Þhð1Þhð2Þ d1 d2 À1 ¼ E½Š hðÞ Ã hðÀÞ þ RðÞ Ã hðÞ Ã hðÀÞ ð8:60Þ Now it is concluded that the doubly stochastic process YðtÞ is wide-sense stationary as well. From this latter expression its power spectral density is easily revealed by Fourier transformation: SYY ð!Þ ¼ jHð!Þj2fE½Š þ Sð!Þg ð8:61Þ The interpretation of this expression is as follows. The first term reflects the filtering of the white shot noise spectrum and is proportional to the mean value of the information signal ðtÞ. Note that E½ðtފ ¼ 0 is meaningless from a physical point of view. The second term represents the filtering of the information signal. An important physical situation where the theory in this section applies is a lightwave that is intensity modulated by an information signal. The detected current in the receiver is produced by photons arriving in the photodetector. This arrival of photons is then modelled as a doubly stochastic process with ðtÞ as the information signal [8]. 8.5 THE RANDOM-PULSE PROCESS Let us further extend the inhomogeneous process that was introduced in the preceding section. A Poisson impulse process consists of a sequence of  functions X XðtÞ ¼ Giðt À tiÞ ð8:62Þ i where the number of events per unit of time are governed by a Poisson distribution and fGig are realizations of the random variable G; i.e. the amplitudes of the different impulses vary randomly and thus are subject to a random gain. In this section we again assume that the Poisson distribution may have a time-variant parameter ðtÞ, which is supposed to be a widesense stationary stochastic process. Each of three random parameters involved is assumed to be independent of all the others. When filtering the process XðtÞ we get the random-pulse process X YðtÞ ¼ Gihðt À tiÞ i ð8:63Þ The properties of this process are again derived in a similar and straightforward way, as presented in Section 8.3.1. Throughout the entire derivation a third random variable is 206 POISSON PROCESSES AND SHOT NOISE involved and the expectation over this variable has to be taken in addition. This leads to the following result for the characteristic function: Z 1  ÈY ðuÞ ¼ exp ðÞfÈG½uhðt À ފ À 1g d À1 ð8:64Þ where ÈGðuÞ is the characteristic function of G. From this another extension of Campbell’s theorem follows: Z1 E½Yðtފ ¼ E½GŠ ðÞhðt À Þ d ZÀ11  2 Y ¼ E½G2Š ðÞh2ðt À Þ d À1 ð8:65Þ ð8:66Þ The second joint characteristic function reads Z1 Z 1  ÉY ðu1; u2Þ ¼ fGðgÞ ðÞfexp½ju1hðt1 À Þ þ ju2hðt2 À ފ À 1g d dg ð8:67Þ À1 À1 where fGðgÞ is the probability density function of G. Also in this case the process YðtÞ appears to be wide-sense stationary and the autocorrelation function becomes RYY ð Þ ¼ E½G2Š E½Š hð Þ Ã hðÀ Þ þ E2½GŠ RðÞ Ã hð Þ Ã hðÀ Þ ð8:68Þ and the power spectral density SYY ð!Þ ¼ ¼ jHð!Þj2 jHð!Þj2 fE½G2Š E½Š þ E2½GŠ &E½G2Š E2½GŠ E2½GŠ S ð!Þg ' E½Š þ Sð!Þ ð8:69Þ The last term will in general comprise the information, whereas the first term is the shot noise spectral density. In applications where the random impulse amplitude G is an amplification generated by an optical or electronic device, the factor E½G2Š=E2½GŠ is called the excess noise factor. It is the factor by which the signal-to-shot noise ratio is decreased compared to the situation where such amplification is absent, i.e. G is constant and equal to 1. Since E½G2Š E2½GŠ ¼ E2½GŠ þ 2G E2½GŠ ¼ 1 þ 2G E2½GŠ > 1 ð8:70Þ the excess noise factor is always larger than 1. Example 8.5: A random-pulse process with the Poisson parameter  ¼ constant is applied to a filter with the impulse response given by Equation (8.42). The G’s are, independently from each other, SUMMARY 207 selected from the set G 2 f1; À1g with equal probabilities. From these data and Campbell’s theorem (Equations (8.65) and (8.66)), it follows that E½GŠ ¼ E½YŠ ¼ 0 E½G2Š ¼ 1 2Y ¼ T ð8:71Þ This reduces Equation (8.69) to just the first term. The power spectrum of the output becomes SYY ð!Þ ¼ 4 sin2 À!T 2 Á !2 ð8:72Þ & There are two main application areas for the processes dealt with in this section. The first example is the current produced by a photomultiplier tube or avalanche photodiode when an optical signal is detected. Each detected photon produces many electrons as a consequence of the internal amplification in the detector device. This process can be modelled as a Poisson impulse process, where G is a random variable with discrete integer amplitude. Looking at Equation (8.70) one may wonder why such amplification is applied when the signal-to-noise ratio is decreased by it. From the first line in this equation it is revealed that the information signal is amplified by E2½GŠ and that is useful, since it raises the signal level with respect to the thermal noise (not taken into account in this chapter), which is dominant over the shot noise in the case of weak signal reception. Secondly, when a radar on board an aircraft flying over the sea transmits pulses, it receives many copies of the transmitted pulse, called clutter. These echoes are randomly distributed in time and amplitude, due to reflections from the moving water waves. Similar reflections are received when flying over land, due to the relative changes in the earth’s surface and eventual buildings. This process may be modelled as a random-pulse process. 8.6 SUMMARY The Poisson distribution is recalled. Subsequently, the characteristic function is defined. This function provides a powerful tool for calculating moments of random variables. When taking the logarithm of the characteristic function, the second characteristic function results and it can be of help calculating moments of Poisson distributions and processes. Based on these functions several properties of the Poisson distribution are derived. The probability density function of the interarrival time is calculated. The homogeneous Poisson process consists of a sequence of unit impulses with random distribution on the time axis and a Poisson distribution of the number of impulses per unit of time. When filtering this process a noise-like signal results, called shot noise. Based on the characteristic function and the second characteristic function several properties of this process are derived, such as the mean value, variance (Campbell’s theorem) and autocorrelation function. From these calculations it follows that the homogeneous Poisson process is a wide-sense stationary process. Fourier transforming the autocorrelation function 208 POISSON PROCESSES AND SHOT NOISE provides the power spectral density. Although part of the shot noise process behaves like white noise, it is not additive but multiplicative. Derivation of the properties of the inhomogeneous Poisson process and the random-pulse process is similar to that of the homogeneous Poisson process. Although our approach emphasizes using the characteristic function to calculate the properties of random point processes, it should be stressed that this is not the only way to arrive at these results. A few application areas of random point processes are mentioned. 8.7 PROBLEMS 8.1 Consider two independent random variables X and Y. The random variable Z is defined as the sum Z ¼ X þ Y. Based on the use of characteristic functions show that the probability density function of Z equals the convolution of the probability density functions of X and Y. 8.2 Calculate the characteristic function of a Gaussian random variable. 8.3 Consider two jointly Gaussian random variables, X and Y, both having zero mean and variance of unity. Using the characteristic functions show that the probability density function of their sum, Z ¼ X þ Y, is Gaussian as well. 8.4 Use the characteristic function to calculate the variance of the waiting time of a Poisson process. 8.5 The characteristic function can be used to calculate the probability density function of a function gðXÞ of a random variable if the transformation Y ¼ gðXÞ is one-to-one. Consider Y ¼ sin X, where X is uniformly distributed on the interval ðÀp=2; p=2Š. (a) Calculate the probability density function of Y. (b) Use the result of (a) to calculate the probability density function of a full sine wave, i.e. Y ¼ sin X, where now X is uniformly distributed on the interval ðÀp; pŠ Hint: extend the sine wave so that X uniformly covers the interval ðÀp; pŠ. 8.6 A circuit comprises 100 components. The circuit fails if one of the components fails. The time to failure of one component is exponentially distributed with a mean time to failure of 10 years. This distribution is the same for all components and the failure of each components is independent of the others. (a) What is the probability that the circuit will be in operation for at least one year without interruption due to failure? (b) What should the mean time to failure of a single component be so that the probability that the circuit will be in operation for at least one year without interruption is 0.9? 8.7 A switching centre has 100 incoming lines and one outgoing line. The arrival of calls is Poisson distributed with an average rate of 5 calls per hour. Suppose that each call lasts exactly 3 minutes. (a) Calculate the probability that an incoming call finds the outgoing line blocked. PROBLEMS 209 (b) The subscribers have a contract that guarantees a blocking probability less than 0.01. They complain that the blocking probability is higher than what was promised in the contract with the provider. Are they right? (c) How many outgoing lines are needed to meet this condition in the contract? 8.8 Visitors enter a museum according to a Poisson distribution with a mean of 10 visitors per hour. Each visitor stays in the museum for exactly 0.5 hour. (a) What is the mean value of the number of visitors present in the museum? (b) What is the variance of the number of visitors present in the museum? (c) What is the probability that there are no visitors in the museum? 8.9 Consider a shot noise process with constant parameter . This process is applied to a filter that has the impulse response hðtÞ ¼ expðÀ tÞ uðtÞ, with uðtÞ the unit step function. (a) Find the mean value of the filtered shot noise process. (b) Find the variance of the filtered process. (c) Find the autocorrelation function of the filtered process. (d) Calculate the power spectral density of the filtered process. 8.10 We want to consider the properties of a filtered shot noise process YðtÞ with constant parameter  ) 1. The problem is that when  ! 1 both the mean and the variance become infinitely large. Therefore we consider the normalized process with zero mean and variance of unity for all  values: ÄðtÞ ¼4 Y ðtÞ À E½Y Y ðtފ ¼ Y ðtÞpÀffiffiffiA B where Z1 A ¼4 hðÞ d ZÀ11 B2 ¼4 h2ð Þ d À1 Apply Equation (8.38) and the series expansion of Equation (8.39) to the process ÄðtÞ to prove that the characteristic function of ÄðtÞ tends to that of a Gaussian random variable (see the outcome of Problem 8.2) and thus the filtered Poisson process approaches a Gaussian process when the Poisson parameter  becomes large. 8.11 The amplitudes of the impulses of a Poisson impulse process are independent and identically distributed with PðGi ¼ 1Þ ¼ 0:6 and PðGi ¼ 2Þ ¼ 0:4. The process is applied to a linear time-invariant filter with a rectangular impulse response of height 10 and duration T. The Poisson parameter  is constant. (a) Find the mean value of the filter output process. (b) Find the autocorrelation function of the filter output process. (c) Calculate the power spectral density of the output process. 8.12 The generation of hole–electron pairs in a photodiode is modelled as a Poisson process due to the random arrival of photons. When the optical wave is modulated by a 210 POISSON PROCESSES AND SHOT NOISE randomly phased harmonic signal this arrival has a time-dependent rate of  ¼ 0½1 þ cosð!0t À Âފ with  a random variable that is uniformly distributed on the interval ð0; 2pŠ, and where the cosine term is the information-carrying signal. Each hole–electron pair creates an impulse of height e, being the electron charge. The photodetector has a random internal gain that has only integer values and that is uniformly distributed on the interval ½0; 10Š. The travel time T in the detector is modelled as an impulse response & hðtÞ ¼ 1; 0 t < T 0; elsewhere: (a) Calculate the mean value of the photodiode current and the power of the detected signal current. (b) Calculate the shot noise variance. (c) Calculate the the signal-to-noise ratio in dB for 0 ¼ 7:5  1012, T ¼ 10À9 and !0 ¼ 2p  0:5  109. References 1. A. Papoulis and S.U Pillai, Probability, Random Variables, and Stochastic Processes, fourth edition, McGraw-Hill, 2002. 2. P.Z. Peebles, Probability, Random Variables, and Random Signal Principles, fourth edition, McGraw-Hill, 2001. 3. C.W. Helstrom, Probability and Stochastic Processes for Engineers, second edition, Macmillan, 1991. 4. W.B. Davenport Jr and J.L. Root, An Introduction to the Theory of Random Signals and Noise, McGraw-Hill, 1958; reissue by IEEE Press, 1987. 5. K.S. Shanmugam and A.M. Breipohl, Random Signals: Detection, Estimation and Data Analysis, Wiley, 1988. 6. L.W. Couch II, Digital and Analog Communication Systems, sixth edition, Prentice-Hall, 2001. 7. A. Papoulis, The Fourier Integral and Its Applications, McGraw-Hill, 1962. 8. W. van Etten and J. van der Plaats, Fundamentals of Optical Fiber Communications, Prentice-Hall, 1991. 9. J. Proakis and M. Salehi, Communication Systems Engineering, second edition, Prentice-Hall, 2002. 10. H. Baher, Analog and Digital Signal Processing, second edition, Wiley, 2001. 11. S. Haykin, Communication Systems, fourth edition, Wiley, 2001. 12. A. Papoulis, Signal Analysis, McGraw-Hill, 1977. 13. A.B. Carlson, P.B. Crilly and J.C. Rutledge, Communication Systems, fourth edition, McGraw-Hill, 2002. 14. J.M. Wozencraft and I.M. Jacobs, Principles of Communication Engineering, Wiley, 1965. 15. C.W. Helstrom, Statistical Theory of Signal Detection, second edition, Pergamon, 1968. 16. L.A. Pipes, Applied Mathematics for Engineers and Physicists, second edition, McGraw-Hill, 1958. 17. D.L. Snyder and M.I. Miller, Random Point Processes in Time and Space, second edition, Springer, 1991. Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd Further Reading Ash, C., The Probability Tutoring Book, IEEE Press, 1993. Bracewell, R., The Fourier Transform and Its Applications, second edition, Cambridge University Press, 1978. Childers, D.G., Probability and Random Processes, Irwin, 1997. Davenport Jr, W.B., Probability and Random Processes, McGraw-Hill, 1970. Feller, W., An Introduction to Probability Theory and Its Applications, Vol. I, third edition, Wiley, 1968. Franks, L.E., Signal Theory, Prentice-Hall, 1969. Gradshteyn, I.S. and Ryzhik, I.M., Table of Integrals, Series and Products, fourth edition, Academic Press, 1980. Leon-Garcia, A., Probability and Random Processes for Electrical Engineers, second edition, Addison- Wesley, 1994. McDonough, R.N. and Whalen, A.D., Detection of Signals in Noise, second edition, Academic Press, 1995. Melsa, J.L. and Cohn, D.L., Decision and Estimation Theory, McGraw-Hill, 1978. Middleton, D., An Introduction to Statistical Communication Theory, McGraw-Hill, 1960; reissue by IEEE Press, 1997. Poor, H.V., An Introduction to Signal Detection and Estimation, second edition, Springer-Verlag, 1994. Proakis, J., Digital Communications, fourth edition, McGraw-Hill, 2003. Schwartz, M., Information Transmission, Modulation, and Noise, fourth edition, McGraw-Hill, 1990. Schwartz, M. and Shaw, L., Signal Processing: Discrete Spectral Analysis, Detection and Estimation, McGraw-Hill, 1975. Thomas, J.B., An Introduction to Statistical Communication Theory, Wiley, 1969. Thomas, J.B., An Introduction to Applied Probability and Random Processes, Krieger, 1981. Van Trees, H.L., Detection, Estimation, and Modulation Theory, Part I, Wiley, 1968. Van Trees, H.L., Detection, Estimation, and Modulation Theory, Part II, Wiley, 1971. Van Trees, H.L., Detection, Estimation, and Modulation Theory, Part III, Wiley, 1971. Ziemer, R.E. and Tranter, W.H., Principles of Communications: Systems, Modulation, and Noise, fifth edition, Wiley, 2002. Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd Appendix A Representation of Signals in a Signal Space A.1 LINEAR VECTOR SPACES In order to facilitate the geometrical representation of signals, we treat them as vectors. Indeed, signals can be considered to behave like vectors, as will be shown in the sequel. For that purpose we recall the properties of linear vector spaces. A vector space is called a linear vector space if it satisfies the following conditions: 1: x þ y ¼ y þ x 2: x þ ðy þ zÞ ¼ ðx þ yÞ þ z 3: ðx þ yÞ ¼ x þ y 4: ð þ Þx ¼ x þ x ðA:1Þ ðA:2Þ ðA:3Þ ðA:4Þ where x and y are arbitrary vectors and and are scalars. In an n-dimensional linear vector space we define a so-called inner product as x Á y ¼4 Xn xiyi i¼1 ðA:5Þ where xi and yi are the elements of x and y, respectively. Two vectors x and y are said to be orthogonal if x Á y ¼ 0. The norm of a vector x is denoted by k x k and we define it by sffiffiffiffiffiffiffiffiffiffiffiffi k x k ¼4 pffiffiffiffiffiffiffiffi xÁx ¼ Xn x2i i¼1 ðA:6Þ Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 216 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE This norm has the following properties: 5: k x k ! 0 6: k x k ¼ 0 () x ¼ 0 7: k x þ y k k x k þ k y k 8: k x k ¼ j j Á k x k ðA:7Þ ðA:8Þ ðA:9Þ ðA:10Þ In general, we can state that the norm of a vector represents the distance from an arbitrary point described by the vector to the origin, or alternatively it is interpreted as the length of the vector. From Equation (A.9) we can readily derive the Schwarz inequality jx Á yj k x k Á k y k ðA:11Þ A.2 THE SIGNAL SPACE CONCEPT In this section we consider signals defined on the time interval ½a; bŠ. As in the case of vectors, we define the inner product of two signals xðtÞ and yðtÞ, but now the definition reads Zb hxðtÞ; yðtÞi ¼4 xðtÞyðtÞ dt a ðA:12Þ Note that using this definition, signals behave like vectors, i.e. they show the properties 1 to 8 as given in the preceding section. This is readily verified by considering the properties of integrals. Let us consider a set of orthonormal signals fiðtÞg, i.e. signals that satisfy the condition  hiðtÞ; jðtÞi ¼ ij ¼4 1; 0; i ¼ j; i 6¼ j ðA:13Þ where ij denotes the well-known Kronecker delta. When all signals of a specific class can exactly be described as a linear combination of the members of such a signal set, we call it a complete orthonormal signal set; here we take the class of square integrable signals. In this case each arbitrary signal of that class is written as Xn sðtÞ ¼ siiðtÞ i¼1 ðA:14Þ and the sequence fsig can also be written as a vector s, where the si are the elements of the signal vector s. Figure A.1 shows a circuit that reconstructs sðtÞ from the set fsig and the orthonormal signal set fiðtÞg; this circuit follows immediately from Equation (A.14). No limitations are placed on the integer n; even an infinite number of elements is allowed. The elements si are found by Zb si ¼ hsðtÞ; iðtÞi ¼ sðtÞiðtÞ dt a ðA:15Þ φ1(t) s1 φ2(t) s2 THE SIGNAL SPACE CONCEPT 217 s(t) ..... ..... ..... φn(t) sn Figure A.1 A circuit that produces the signal sðtÞ from its elements fsig in the signal space φ1(t) b (.) dt s1 a φ2(t) s(t) b (.) dt s2 a ..... ..... ..... φn(t) b (.) dt sn a Figure A.2 A circuit that produces the signal space elements fsig from the signal sðtÞ Therefore, when we construct a vector space using the vector s, we have a geometrical representation of the signal sðtÞ. Along the ith axis we imagine the function iðtÞ; i.e. the set fiðtÞg is taken as the basis for the signal space. In fact, si indicates to what extent iðtÞ contributes to sðtÞ. From Equation (A.15) a circuit is derived that produces the elements fsig representing the signal sðtÞ in the signal space fiðtÞg; the circuit is given in Figure A.2. The inner product of a signal vector with itself has an interesting interpretation: Zb X hsðtÞ; sðtÞi ¼ s2ðtÞ dt ¼ k s k2 ¼ s2i ¼ Es a i ðA:16Þ 218 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE with Es the energy of the signal sðtÞ. From this equation it is concluded that the length of a vector in the signal space equals the square root of the signal energy. For purposes of detection it is important to establish that the distance between two signal vectors represents the square root of the energy of the difference of the two signals involved. This concept of signal spaces is in fact a generalization of the well-known Fourier series expansion of signals. Example A.1: As an example let us consider the harmonic signal sðtÞ ¼ Refa exp½jð!0t þ ފg, where RefÁg is the real part of the expression in the braces. This signal is written as sðtÞ ¼ a cos cos !0t À a sin sin !0t ðA:17Þ pffiffi pffiffiffi pffiffi pffiffiffi As the orthonormal signal set we consider f 2 cosð!0tÞ= T; À 2 sinð!0tÞ= Tg and the time interval ½a; bŠ is taken as ½0; TŠ, with T ¼ k pÂffi2ffiffip=!0 anpd ffikffi ipntffieffiffiger. In thpisffiffisignal space, the signal sðtÞ is represented by the vector s ¼ ½ Ta cos = 2; Ta sin = 2Š. In fact, we have introduced in this way an alternative for the signal representation of harmonic signals in the complex plane. The elements of the vector s are recognized as the well-known quadrature I and Q signals. & A.3 GRAM–SCHMIDT ORTHOGONALIZATION When we have an arbitrary set ffiðtÞg of, let us say, N signals, then in general these signals will not be orthonormal. The Gram–Schmidt method shows us how to transform such a set into an orthonormal set, provided the members of the set ffiðtÞg are linearly independent, i.e. none of the signal fiðtÞ can be written as a linear combination of the other signals. The first member of the orthonormal set is simply constructed as 1ðtÞ ¼4 pffiffiffiffiffifffiffi1ffiðffiffitffiffiÞffiffiffiffiffiffiffiffiffiffiffi hf1ðtÞ; f1ðtÞi ðA:18Þ In fact, the signal f1ðtÞ is normalized to the square root of its energy, or equivalently to the length of its corresponding vector in the signal space. The second member of the orthonormal set is constructed by taking f2ðtÞ and subtracting from this f2ðtÞ the part that is already comprised of 1ðtÞ. In this way we arrive at the intermediate signal g2ðtÞ ¼4 f2ðtÞ À hf2ðtÞ; 1ðtÞi1ðtÞ ðA:19Þ Due to this operation the signal g2ðtÞ will be orthogonal to 1ðtÞ. The functions 1ðtÞ and 2ðtÞ will become orthonormal if we construct 2ðtÞ from g2ðtÞ by normalizing it by its own length: 2ðtÞ ¼ pffiffiffiffiffiffigffiffi2ffiffiðffiffitffiÞffiffiffiffiffiffiffiffiffiffiffiffi hg2ðtÞ; g2ðtÞi ðA:20Þ THE REPRESENTATION OF NOISE IN SIGNAL SPACE 219 Proceeding in this way we construct the kth intermediate signal by gkðtÞ ¼4 fkðtÞ À X kÀ1 hfk ðtÞ; jðtÞijðtÞ j¼1 ðA:21Þ and by proper normalization we arrive at the kth member of the orthonormal signal set kðtÞ ¼4 pffiffiffiffiffiffigffiffikffiffiðffiffitffiÞffiffiffiffiffiffiffiffiffiffiffiffi hgkðtÞ; gkðtÞi ðA:22Þ This procedure is continued until N orthonormal signals have been constructed. In case there are linear dependences, the dimensionality of the orthonormal signal space will be lower than N. The orthonormal space that results from the Gram–Schmidt procedure is not unique; it will depend on the order in which the above described procedure is executed. Nevertheless, the geometrical signal constellation will not alter and the lengths of the vectors are invariant to the order chosen. Example A.2: Consider the signal set f fiðtÞg given in Figure A.3(a). Since the norm of f1ðtÞ is unity, we conclude that 1ðtÞ ¼ f1ðtÞ. Moreover, from the figure it is deduced that the inner product h f2ðtÞ; 1ðtÞi ¼ 0 and h f2ðtÞ; f2ðtÞi ¼ 4, which means that 2ðtÞ ¼ f2ðtÞ=2. Once this is known, it is easily verified that h f3ðtÞ; 1ðtÞi ¼ 1 2 and h f3ðtÞ; 2ðtÞi ¼ 0, from which it follows that g3ðtÞ ¼ f3ðtÞ À 1ðtÞ=2. The set of functions fgiðtÞg has been depicted in Figure A.3(b). From this figure we calculate k g3 k2¼ 14, so that 3ðtÞ ¼ 2g3ðtÞ, and finally from all those results the signal set fiðtÞg, as given in Figure A.3(c), can be constructed. In this example the given functions do not show linear dependence and therefore the set fiðtÞg contains as many functions as the given set f fiðtÞg. & A.4 THE REPRESENTATION OF NOISE IN SIGNAL SPACE In this section we will confine our analysis to the widely used concept of wide-sense stationary, zero mean, white, Gaussian noise. A sample function of the noise is denoted by NðtÞ and the spectral density is N0=2. We construct the noise vector n in the signal space, where the elements of this noise vector are defined by ni ¼ hNðtÞ; iðtÞi ðA:23Þ Since this integration is a linear operation, these noise elements will also show a Gaussian distribution and it will be clear that the mean of ni equals zero, for all i. When, besides these data, the cross-correlations of the different noise elements are determined, the noise 220 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE f1(t) 1 0 f2(t) 2 1t 0½ f3(t) 1 1 t 0¼ ¾ 1t g1(t) 1 0 −2 (a) g2(t) 2 1t 0½ g3(t) 1 ½ 1 t −½ 0 ¼ ¾ t −2 (b) φ1(t) φ2(t) φ3(t) 1 1 1 1 1 0 1t 0½ t 0¼ ¾ t −1 −1 (c) Figure A.3 The construction of a set of orthonormal functions fiðtÞg from a set of given functions ffiðtÞg using the Gram–Schmidt orthogonalization procedure elements are completely specified. Those cross-correlations are found to be Z b Zb  E½ninjŠ ¼ E NðtÞ iðtÞ dt NðÞ jð Þ d a 2ZZb a 3 ¼ E4 NðtÞ NðÞ iðtÞ jðÞ dt d5 a ZZb ¼ E½NðtÞ NðފiðtÞjðÞ dt d ðA:24Þ a In this expression the expectation represents the autocorrelation function of the noise, which reads RNNðt; Þ ¼ ðt À ÞN0=2. This is inserted in the last equation to arrive at ZZb E½ninjŠ ¼ N0 2 ðt À ÞiðtÞjðÞ dt d ¼ N0 2 Za b a iðtÞjðtÞ dt ðA:25Þ THE REPRESENTATION OF NOISE IN SIGNAL SPACE 221 Remembering that the set fiðtÞg is orthonormal, the following cross-correlations result 8 E½ninjŠ ¼ < : N0 2 0; ; for i ¼ j for i 6¼ j ðA:26Þ From this equation it is concluded that the different noise elements ni are uncorrelated and, since they are Gaussian with zero mean, they are independent. Moreover, all noise elements show the same variance of N0=2. This simple and symmetric result is another interesting feature of the orthonormal signal spaces as introduced in this appendix. A.4.1 Relevant and Irrelevant Noise When considering a signal that is disturbed by noise we want to construct a common signal space to describe both the signal and noise in the same space. In that case we construct a signal space to completely describe all possible signals involved. When we want to attempt to describe the noise NðtÞ using that signal space, it will, as a rule, be inadequate to completely characterize the noise. In that case we split the noise into one part NrðtÞ that is projected on to the signal space, called the relevant noise, and another part NiðtÞ that is orthogonal to the space set up by the signals, called the irrelevant noise. Thus, the relevant noise is given by the vector nr with components Zb nr;i ¼ NðtÞiðtÞ dt a ðA:27Þ By definition the irrelevant noise reads NiðtÞ ¼4 NðtÞ À NrðtÞ ðA:28Þ Next we will show that the irrelevant noise part is orthogonal to the signal space. For this purpose let us consider a signal sðtÞ and an orthonormal signal space fiðtÞg. Let us suppose that sðtÞ can completely be described as a vector in this signal space. The inner product of the irrelevant noise NiðtÞ and the signal reads Zb Zb NiðtÞsðtÞ dt ¼ fNðtÞ À NrðtÞgsðtÞ dt a a Zb Zb ¼ NðtÞsðtÞ dt À NrðtÞsðtÞ dt a a Zb X Z bX ¼ NðtÞ skkðtÞ dt À nr; kkðtÞsðtÞ dt a k ak Z bX X ¼ skNðtÞkðtÞ dt À nr; ksk ak k X X ¼ sknr; k À nr; ksk ¼ 0 k k ðA:29Þ 222 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE For certain applications, for instance optimum detection of a known signal in noise, it appears that the irrelevant part of the noise may be discarded. A.5 SIGNAL CONSTELLATIONS In this section we will present the signal space description of a few signals that are often met in practice. The signal space with indications of possible signal realizations is called a signal constellation. In detection, the error probability appears always to be a function of Ed=N0, where Ed is the energy of the difference of signals. However, we learned in Section A.2 that this energy is in the signal space represented by the squared distance of the signals involved. Considering two signals siðtÞ and sjðtÞ, their squared distance is written as Zb di2j ¼ k si À sj k2¼ ½siðtÞ À sjðtފ2 dt ¼ Ei þ Ej À 2Eij a ðA:30Þ where Ei and Ej are the energies of the corresponding signals and Eij represents the inner product of the signals. In specific cases where Ei ¼ Ej ¼ E for all i and j, Equation (A.30) is written as di2j ¼ 2Eð1 À ijÞ ðA:31Þ with the cross-correlations ij defined by ij ¼4 k si si k Á Á sj k sj k ðA:32Þ A.5.1 Binary Antipodal Signals Consider the two rectangular signals rffiffiffi s1ðtÞ ¼ Às2ðtÞ ¼ E; 0 T t T ðA:33Þ and their bandpass equivalents rffiffiffiffiffi s1ðtÞ ¼ Às2ðtÞ ¼ 2E T cos !0t; 0 t T ðA:34Þ with T ¼ n  2p=!0 and n integer. The cross-correlation of those signals equals À1 and the distance between the two signals is pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffi d12 ¼ 2Eð1 À 12Þ ¼ 2 E ðA:35Þ SIGNAL CONSTELLATIONS 223 s2 s1 E E Figure A.4 The signal constellation of the binary antipodal signals The space is opneffiffi-ffidimensional, psinffifficffi e the two signals involved are dependent. The signal vectors are s1 ¼ ½ EŠ and s2 ¼ ½À EŠ. This signal set is called an antipodal signal constellation and is depicted in Figure A.4. A.5.2 Binary Orthogonal Signals Next we consider the signal set rffiffiffiffiffi s1ðtÞ ¼ 2E T cos !0t; rffiffiffiffiffi s2ðtÞ ¼ 2E T sin !0t; 0tT 0tT ðA:36Þ ðA:37Þ where either T ¼ np=!0 (with n integer) or T ) 1=!0, so that 12 ¼ 0 or 12 % 0, respectively. Due to this property those signaplsffiffiffiare called orthogonpalffiffiffisignals. In this case the signal space is two-dimensional and s1 ¼ ½ E; 0Š, whilpe sffiffi2ffiffiffi¼ ½0; EŠ. It is easily verified that the distance between the signals amounts to d12 ¼ 2E. The signal constellation is given in Figure A.5. s2 d 12 = 2E s1 Figure A.5 The signal constellation of the binary orthogonal signals 224 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE s2 E s3 s1 s4 s5 s3 E s2 s1 s6 s8 s4 s7 (a) (b) Figure A.6 The signal constellation of M-phase signals A.5.3 Multiphase Signals pffiffiffi In the multiphase case all possible signal vectors are on a circle with radius E. This corresponds to the M-ary phase modulation. The signals are represented by (rffiffiffiffiffi  ) siðtÞ ¼ Re 2E exp T j!0t þ jði À 1Þ 2p M ; for i ¼ 1; 2; . . . ; M; 0 tT ðA:38Þ with the same requirement for the relation between T and !0 as in the former section. The signal vectors are given by si ¼ pffiffiffi E cos ði À 1Þ2p M ; pffiffiffi E sin ði À 1Þ2p M ðA:39Þ In Figure A.6 two examples are depicted, namely M ¼ 4 in Figure A.6(a) and M ¼ 8 in Figure A.6(b). The case of M ¼ 4 can be considered as a pair of two orthogonal signals; namely the pair of vectors ½s1; s3Š is orthogonal to the pair ½s2; s4Š. For that reason this is a special case of the biorthogonal signal set, which is dealt with later on in this section. This orthogonality is used in QPSK modulation. A.5.4 Multiamplitude Signals The multiamplitude case is a straightforward extension of the antipodal signal constellation. The extension is in fact a manifold, let us say M, of signal vectors on the one-dimensional axis (see Figure A.4). In most applications these points are equidistant. SIGNAL CONSTELLATIONS 225 (a) (b) Figure A.7 The signal constellation of QAM signals: (a) rectangular distribution; (b) circular distribution A.5.5 QAM Signals A QAM (quadrature amplitude modulated) signal is a signal where both the amplitude and phase are modulated. In that sense it is a combination of the multiamplitude and multiphase modulated signal. Different constellations are possible; two of them are depicted in Figure A.7. Figure A.7(a) shows a rectangular grid of possible signal vectors, whereas in the example of Figure A.7(b) the vectors are situated on circles. This signal constellation is used in such applications as high-speed telephone modems, cable modems and digital distribution of audio and video signals over CATV networks. A.5.6 M-ary Orthogonal Signals The M-ary orthogonal signal set is no more no less than an M-dimensional extension of the binary orthogonal signal set; i.e. in this case the signal space has M dimensions and all possible signals are orthogonal to all others. For M ¼ 3 and assuming that all signals bear the same energy, the signal vectors are pffiffiffi s1 ¼ ½ E; 0; 0Š pffiffiffi s2 ¼ ½0; E; 0Š pffiffiffi s3 ¼ ½0; 0; EŠ ðA:40Þ This signal sept ffihffiffiffiaffis been illustrated in Figure A.8. The distance between two arbitrary signal pairs is d ¼ 2E. A.5.7 Biorthogonal Signals An M-ary biorthogonal signal set is constructed from an M=2-ary orthogonal signal set by simply adding the negatives of all the orthogonal signals. It will be clear that the dimension 226 APPENDIX A: REPRESENTATION OF SIGNALS IN A SIGNAL SPACE s2 2E 2E s1 2E s3 Figure A.8 The signal constellation of the M-ary orthogonal signal set for M ¼ 3 s2 −s 1 −s 2 s2 −s3 −s 1 s1 s1 s3 −s 2 (a) (b) Figure A.9 The signal constellation of the M-ary biorthogonal signal set for (a) M ¼ 4 and (b) M ¼ 6 of the signal space remains as M=2. The result is given in Figure A.9(a) for M ¼ 4 and in Figure A.9(b) for M ¼ 6. A.5.8 Simplex Signals To explain the simplex signal set we start from a set of M orthogonal signals ffiðtÞg with the vector presentation ffig. We determine the mean of this signal set f ¼ 1 M X M fi i¼1 ðA:41Þ PROBLEMS 227 s2 2E 2E s2 s1 s1 s3 (a) (b) Figure A.10 The simplex signal set for (a) M ¼ 2 and (b) M ¼ 3 In order to arrive at the simplex signal set, this mean is subtracted from each vector of the orthogonal set: si ¼ fi À f; i ¼ 1; 2; . . . ; M ðA:42Þ In fact this operation means that the origin is translatedptffioffiffiffiffithe point f. Therefore the distance between any pair of possible vectors si remains d ¼ 2E. Figure A.10 shows the simplex signals for M ¼ 2 and M ¼ 3. Note that the dimensionality is reduced by 1 compared to the starting orthogonal set. Due to the transformation the signals are no longer orthogonal. On the other hand, it will be clear that the mean signal energy has decreased. Since the distance between signal pairs are the same as for orthogonal signals, the simplex signal set is able to realize communication with the same quality (i.e. error probability) but using less energy per bit and thus less power. A.6 PROBLEMS A.1 Derive the Schwarz inequality (A.11) from the triangular inequality (A.9). A.2 Use the properties of integration and the definition of Equation (A.12) to show that signals satisfy the properties of vectors as given in Section A.1. A.3 Use Equation (A.19) to show that g2ðtÞ is orthogonal to 1ðtÞ. A.4 Show that for binary orthogonal signals one of the two given relations between T and !0 is required for orthogonality. A.5 Calculate the distance between adjacent signal vectors, i.e. the minimum distance, for the M-ary phase signal. A.6 Calculate the energy of the signals from the simplex signal set, expressed in terms of the energy in the signals of the M-ary orthogonal signal set. A.7 Calculate the cross-correlation between the various signals of the simplex signal set. Appendix B Attenuation, Phase Shift and Decibels The transfer function of a linear time-invariant system, denoted by Hð!Þ, is in general a complex function of !. It is indicative of the manner in which time harmonic signals (sine and cosine) propagate through such a system. The modulus of Hð!Þ is the factor by which the amplitude of the incoming harmonic signal is scaled and the argument of Hð!Þ indicates the phase shift introduced by the system. We then can write Hð!Þ ¼ exp½Àað!Þ À jbð!ފ ¼ exp½Àað!ފ exp½Àjbð!ފ ¼ jHð!Þj exp½Àjbð!ފ ðB:1Þ where að!Þ ¼ ln 1 jHð!Þj ðB:2Þ is the attenuation of the system in neper (abbreviated as Np) and bð!Þ is the phase shift introduced by the system. The neper is an old unit that is not used in practice anymore. A more common measure of attenuation is the decibel (abbreviated as dB). The attenuation in dB is defined as adð!Þ ¼ 20 log10 1 jHð!Þj ðB:3Þ From the relation log10 x ¼ ln x ln 10 we conclude that adð!Þ ¼ 20 1 ln 10 ln 1 jHð!Þj ¼ 20  0:4343 ln 1 jHð!Þj ¼ 8:686 að!Þ ðB:4Þ ðB:5Þ Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 230 APPENDIX B: ATTENUATION, PHASE SHIFT AND DECIBELS From this it follows that 1 Np is equivalent to 8.686 dB. Because the neper is used infrequently, we will drop the subscript ‘d’ and henceforth speak about attenuation exclusively in terms of dB and denote this as a. The transfer function Hð!Þ is usually defined as a ratio of the output voltage (or current) and the input voltage (or current). If we look at the power ratio between the input and output, it is evident that the powers are proportional to the square of the voltages or currents, i.e. jVij2 Pi Po ¼ jIij2Ri jIoj2Ro ¼ Ri jVoj2 Ro ðB:6Þ where the indices ‘o’ and ‘i’ indicate that the quantity is related to the output and input, respectively. The quantities Ro;i give the real part of the impedances. If we assume the situation where Ro ¼ Ri, which is often the case, then the power attenuation is found as a ¼ 20  log10 jVij jVoj ¼ 20  log10 jIij jIoj ¼ 10  log10 Pi Po ðB:7Þ The advantage in using a logarithmic measure for the ratio of magnitudes between the inputs and outputs lies in the fact that in series connections of systems, the attenuation in each of the systems expressed in dB needs only to be summed in order to obtain the total attenuation in dB. Furthermore, we mention that the decibel is also used to express the absolute levels of power, current and voltage. The following notations are among others in use:  dBW – power with respect to P0 ¼ 1 W  dBm – power with respect to P0 ¼ 1 mW  dBV – voltage with respect to V0 ¼ 1 V. Definitions and general properties of logarithms are given in Appendix C, Section C.6. Table B.1 presents a list of linear power ratios and the corresponding amount of dBs. Table B.1 List of dB values for a given linear power ratio Linear ratio dB 0.25 À6 0.5 À3 1 0 2 3 4 6 10 10 100 20 1000 30 Appendix C Mathematical Relations C.1 TRIGONOMETRIC RELATIONS cosð Æ Þ ¼ cos cos Ç sin sin sinð Æ Þ ¼ sin cos Æ cos sin cosð Æ pÞ ¼ Ç sin 2 sinð Æ pÞ ¼ Æ cos 2 cos 2 ¼ cos2 À sin2 sin 2 ¼ 2 sin cos cos ¼ 1 ½expðj Þ þ expðÀj ފ 2 sin ¼ 1 ½expðj Þ À expðÀj ފ 2j cos cos ¼ 1 ½cosð À Þ þ cosð þ ފ 2 sin sin ¼ 1 ½cosð À Þ À cosð þ ފ 2 sin cos ¼ 1 ½sinð À Þ þ sinð þ ފ 2 cos2 ¼ 1 ð1 þ cos 2 Þ 2 sin2 ¼ 1 ð1 À cos 2 Þ 2 cos3 ¼ 3 cos þ 1 cos 3 4 4 sin3 ¼ 3 sin À 1 sin 3 4 4 Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd ðC:1Þ ðC:2Þ ðC:3Þ ðC:4Þ ðC:5Þ ðC:6Þ ðC:7Þ ðC:8Þ ðC:9Þ ðC:10Þ ðC:11Þ ðC:12Þ ðC:13Þ ðC:14Þ ðC:15Þ 232 APPENDIX C: MATHEMATICAL RELATIONS cos4 ¼ 3 þ 1 cos 2 þ 1 cos 4 82 8 sin4 ¼ 3 À 1 cos 2 þ 1 cos 4 82 8 A cos À B sin ¼ R cosð þ Þ where pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi R ¼ A2 þ B2;  ¼ arctanðB=AÞ; A ¼ R cos  B ¼ R sin  C.2 DERIVATIVES y ¼ yðxÞ y¼a y¼x y ¼ xn y ¼ ln x; x > 0 y ¼ log x; x > 0 y ¼ sin x y ¼ cos x y ¼ tan x y ¼ cot x y ¼ exp x y ¼ ax y ¼ arcsin x; jxj < 1 y ¼ arccos x; jxj < 1 y ¼ arctan x y ¼ sinðax þ bÞ y0 ¼ dy=dx y0 ¼ 0; a constant y0 ¼ 1 y0 ¼ nxnÀ1 y0 ¼ 1=x y0 ¼ ð1=xÞ log e y0 ¼ cos x y0 ¼ Àsin x y0 ¼ 1= cos2 x y0 ¼ À1= sin2 x y0 ¼ exp x y0 ¼ ax plnffiaffiffiffiffiffiffiffiffiffiffiffi y0 ¼ 1= p1 Àffiffiffiffiffixffiffi2ffiffiffiffiffi y0 ¼ À1= 1 À x2 y0 ¼ 1=ð1 þ x2Þ y0 ¼ a cosðax þ bÞ C.2.1 Rules for Differentiation y¼uþv y ¼ uv y ¼ uvw y ¼ u=v y ¼ uv y0 ¼ u0 þ v0 y0 ¼ u0v þ uv0 y0 ¼ u0vw þ uv0w þ uvw0 y0 ¼ ðu0v À uv0Þ=v2 y0 ¼ uvðv0 ln u þ u0v=uÞ C.2.2 Chain Rule y ¼ f ðzÞ; z ¼ gðxÞ ! y0 ¼ dy  dz dz dx ðC:16Þ ðC:17Þ ðC:18Þ ðC:19Þ ðC:20Þ ðC:21Þ ðC:22Þ ðC:23Þ ðC:24Þ ðC:25Þ ðC:26Þ ðC:27Þ ðC:28Þ ðC:29Þ ðC:30Þ ðC:31Þ ðC:32Þ ðC:33Þ ðC:34Þ ðC:35Þ ðC:36Þ ðC:37Þ ðC:38Þ ðC:39Þ INDEFINITE INTEGRALS 233 C.2.3 Stationary Points maximum if : minimum if : point of inflection if : f 0ðxÞ ¼ 0 and f 00ðxÞ < 0 f 0ðxÞ ¼ 0 and f 00ðxÞ > 0 f 00ðxÞ ¼ 0 and f 000ðxÞ 6¼ 0 ðC:40Þ ðC:41Þ ðC:42Þ C.3 INDEFINITE INTEGRALS C.3.1 Basic Integrals Z f 0ðxÞ dx ¼ f ðxÞ Zb f ðxÞ dx ¼ FðbÞ À FðaÞ; Za Z xn dx ¼ xnþ1 nþ1 1 dx ¼ ln jxj Zx Z if f ðxÞ dx ¼ FðxÞ sin x dx ¼ Àcos x Z cos x dx ¼ sin x Z tan x dx ¼ Àln cos x Z pffiffiffi1ffiffiffiffiffiffiffiffiffi dx ¼ arcsin x ¼ p À arccos x; Z Z 1 À x2 2 1 1 þ x2 dx ¼ arctan x ¼ p 2 À arccot x jxj < 1 exp x dx ¼ exp x Z ax dx ¼ ax ; Z ln a a>0 sinh x dx ¼ cosh x Z cosh x dx ¼ sinh x Z 1 dx ¼  ln tan x Z sin x 2 Z 1 cos2 x dx ¼ tan x 1 sin2 x dx ¼ À cot x ðC:43Þ ðC:44Þ ðC:45Þ ðC:46Þ ðC:47Þ ðC:48Þ ðC:49Þ ðC:50Þ ðC:51Þ ðC:52Þ ðC:53Þ ðC:54Þ ðC:55Þ ðC:56Þ ðC:57Þ ðC:58Þ 234 APPENDIX C: MATHEMATICAL RELATIONS C.3.2 Integration by Parts Z Z Z Z u dv ¼ u v À v du or u v0 dx ¼ u v À u0 v dx Z Z example : ln x dx ¼ x ln x À x 1 dx ¼ xðln x À 1Þ x ðC:59Þ C.3.3 Rational Algebraic Functions Z ða þ bxÞn dx ¼ ða þ bðn bxÞnþ1 þ 1Þ ; n>0 Z a 1 þ bx dx ¼ 1 b ln ja þ bxj Z ða 1 þ bxÞn dx ¼ bðn À À1 1Þða þ bxÞnÀ1 ; n>1 Z   ax2 1 þ bx þ c dx ¼ pffiffiffiffiffiffi2ffiffiffiffiffiffiffiffiffiffiffi arctan ¼ 4ac À b2 pffiffiffiffiffiffi1ffiffiffiffiffiffiffiffiffiffiffi b2 À 4ac ln 22aaxx p2ffiffiaffiffixffiffiffiffiþffiffiffiffibffiffiffiffiffi ; þ þ 4ac bÀ bþ ppÀffibffibbffiffiffiffi22ffiffi2ffiffiffiffiÀÀffiffiffiffiffiffiffiffiffiffi44ffiffiffiffiaaffiffiffifficcffiffiffiffi ; ðC:60Þ ðC:61Þ ðC:62Þ for b2 < 4ac for b2 > 4ac ¼ À2 2ax þ b ; for b2 ¼ 4ac Z Z ax2 x þ bx þ c dx ¼ 1 2a ln jax2 þ bx þ cj À b 2a 1 ax2 þ bx þ c dx Z  a2 1 þ b2x2 dx ¼ 1 ab arctan bx a Z a2 x þ x2 dx ¼ 1 lnða2 2 þ x2Þ Z a2 x2 þ b2x2 dx ¼ x b2 À a b3  bx arctan a Z ða2 1 þ x2Þ2 dx ¼ x 2a2ða2 þ x2Þ þ 1 2a3 x arctan a Z ða2 x þ x2Þ2 dx ¼ À1 2ða2 þ x2Þ Z ða2 x2 þ x2Þ2 dx ¼ Àx 2ða2 þ x2Þ þ 1 x arctan 2a a Z ða2 1 þ x2Þ3 dx ¼ x 4a2ða2 þ x2Þ2 þ 3x 8a4ða2 þ x2Þ þ 3 8a5 x arctan a ðC:63Þ ðC:64Þ ðC:65Þ ðC:66Þ ðC:67Þ ðC:68Þ ðC:69Þ ðC:70Þ ðC:71Þ INDEFINITE INTEGRALS 235 Z ða2 x2 þ x2Þ3 dx ¼ Àx 4ða2 þ x2Þ2 þ x 8a2ða2 þ x2Þ þ 1 8a3 x arctan a Z Z ða2 x4 þ x2Þ3 dx ¼ a2x 4ða2 þ x2Þ2 À 5x 8ða2 þ x2Þ þ 3 x 8a arctan a ða2 1 þ x2Þ4 dx ¼ x 6a2ða2 þ x2Þ3 þ 5x 24a4ða2 þ x2Þ2 þ 5x 16a6ða2 þ x2Þ Z þ 5 16a7 x arctan a ða2 x2 þ x2Þ4 dx ¼ Àx 6ða2 þ x2Þ3 þ x 24a2ða2 þ x2Þ2 þ x 16a4ða2 þ x2Þ Z þ 1 16a5 x arctan a ða2 x4 þ x2Þ4 dx ¼ a2x 6ða2 þ x2Þ3 À 7x 24ða2 þ x2Þ2 þ x 16a2ða2 þ x2Þ Z a4 1 þ x4 dx ¼ 8paffi23ffiþl1n61axx322aþÀrctaaapxxnppffiffiaxffi2ffi2ffiffiþþ a2 a2  þ  pffiffi x2 2 arctan a À  1 þ 2 arctan x 2 þ 1 a Z a4 x2 þ x4 dx ¼ pffiffi 2 8a  À x2 ln þ pffiffi axp2ffiffi x2Àpaffixffi 2 þ a2 þ a2  þ 2  pffiffi x2 arctan a À  1 þ 2 arctan x 2 þ 1 a ðC:72Þ ðC:73Þ ðC:74Þ ðC:75Þ ðC:76Þ ðC:77Þ ðC:78Þ C.3.4 Trigonometric Functions Z cos x dx ¼ sin x Z Z x cosðaxÞ dx ¼ 1 a2 ½cosðaxÞ þ ax sinðaxފ Z x2 cosðaxÞ dx ¼ 1 a3 ½2ax cosðaxÞ þ ða2x2 À 2Þ sinðaxފ sin x dx ¼ À cos x Z Z x sinðaxÞ dx ¼ 1 a2 ½sinðaxÞ À ax cosðaxފ x2 sinðaxÞ dx ¼ 1 a3 ½2ax sinðaxÞ À ðax2 À 2Þ cosðaxފ ðC:79Þ ðC:80Þ ðC:81Þ ðC:82Þ ðC:83Þ ðC:84Þ 236 APPENDIX C: MATHEMATICAL RELATIONS Reduction formulae: Z sinn x dx ¼ À1 sinnÀ1 x cos x þ n À 1 Z sinnÀ2 x dx Z cosn x dx ¼ n 1 cosnÀ1 x sin x þ n n À1 Z cosnÀ2 x dx Z n Zn tann x dx ¼ n 1 À 1 tannÀ1 x À tannÀ2 x dx; n 6¼ 1 ðC:85Þ ðC:86Þ ðC:87Þ C.3.5 Exponential Functions Z expðaxÞ dx ¼ expðaxÞ ; a real or complex a Z   x expðaxÞ dx ¼ expðaxÞ x a À 1 a2 ; a real or complex Z x2 expðaxÞ dx ¼ expðaxÞx2 a À 2x a2 þ  2 a3 ; a real or complex Z x3 expðaxÞ dx ¼ expðaxÞx3 a À 3x2 a2 þ 6x a3 À  6 a4 ; a real or complex Z x expðax2Þ dx ¼ 1 expðax2Þ 2a Z expðaxÞ sinðbxÞ dx ¼ expðaxÞ a2 þ b2 ½a sinðbxÞ À b cosðbxފ Z expðaxÞ cosðbxÞ dx ¼ expðaxÞ a2 þ b2 ½a cosðbxÞ þ b sinðbxފ ðC:88Þ ðC:89Þ ðC:90Þ ðC:91Þ ðC:92Þ ðC:93Þ ðC:94Þ Reduction formula: Z Z exp xn x dx ¼ Àðn À 1Þ exp x xnÀ1 þ ðn À 1Þ exp x xnÀ1 dx ðC:95Þ C.4 DEFINITE INTEGRALS Z 1 expðÀax2Þ dx ¼ rffiffiffi p; a>0 ZÀ11 ZÀ11 expðÀa2x2 þ x2 expðÀax2Þ a pffiffiffi bxÞ dx ¼ p a rffiffiffi dx ¼ 1 p;  b2 exp 4a2 a>0  ; 0 4a a a>0 ðC:96Þ ðC:97Þ ðC:98Þ Z 1 sin x dx ¼ p Z0 x 1sin x2 2 dx ¼ p Z0 Z0 Z0 0 1 1 1 x 2 x sinðaxÞ b2 þ x2 dx ¼ p 2 cosðaxÞ b2 þ x2 dx ¼ p 2b cosðaxÞ ðb2 À x2Þ2 dx ¼ expðÀabÞ; expðÀabÞ; p 4b3 ½sinðabÞ a> a> À ab 0; b > 0 0; b > 0 cosðabފ; a > 0; b > 0 SERIES 237 ðC:99Þ ðC:100Þ ðC:101Þ ðC:102Þ ðC:103Þ C.5 SERIES Taylor expansion: f ðx þ aÞ ¼ f ðxÞ þ af 0ðxÞ þ a2 2! f 00ðxÞ þ Á Á Á þ an n! f ðnÞðxÞ þ Á Á Á McLaurin expansion: f ðxÞ ¼ f ð0Þ þ xf 0ð0Þ þ x2 2! f 00ð0Þ þ Á Á Á þ xn n! f ðnÞð0Þ þ Á Á Á exp x ¼ 1 þ x þ 1 2! x2 þ 1 3! x3 þ Á Á Á sin x ¼ x À 1 3! x3 þ 1 5! x5 À Á Á Á cos x ¼ 1 À 1 2! x2 þ 1 4! x4 À Á Á Á tan x ¼ x þ 1 x3 þ 2 x5 þ Á Á Á 3 15 arcsin x ¼ x þ 1 x3 þ 3 x5 þ Á Á Á 6 40 arctan x ¼ x À 1 x3 þ 1 x5 À Á Á Á ; jxj < 1 35 sinc x ¼ 1 À 1 3! x2 þ 1 5! x4 À Á Á Á lnð1 þ xÞ ¼ x À 1 x2 þ 1 x3 À Á Á Á 23 ð1 þ xÞn ¼ 1 þ nx þ nðn À 2! 1Þ x2 þ nðn À 1Þðn 3! À 2Þ x3 þ Á Á Á ; XN n ¼ NðN þ 1Þ n¼1 2 jnxj < 1 ðC:104Þ ðC:105Þ ðC:106Þ ðC:107Þ ðC:108Þ ðC:109Þ ðC:110Þ ðC:111Þ ðC:112Þ ðC:113Þ ðC:114Þ ðC:115Þ 238 APPENDIX C: MATHEMATICAL RELATIONS XN n2 ¼ NðN þ 1Þð2N þ 1Þ n¼1 6 XN n3 ¼ N2ðN þ 1Þ2 n¼1 4 XN n¼0 xn ¼ 1 À xNþ1 1Àx XN n¼0 N! n!ðN À nÞ! xnyNÀn ¼ ðx þ yÞN X 1 n¼0 axn ¼ 1 a À x ; jxj < 1 C.6 LOGARITHMS Definitions: If 10x ¼ y; then log y ¼4 x; y ! 0 If exp x ¼ y; then ln y ¼4 x; y ! 0 Properties: logðabÞ ¼ log a þ log b a log ¼ log a À log b b lnðabÞ ¼ ln a þ ln b a ln ¼ ln a À ln b b log ab ¼ b log a ln ab ¼ b ln a log a ¼ ln a lnð10Þ ¼ 0:4343  ln a ðC:116Þ ðC:117Þ ðC:118Þ ðC:119Þ ðC:120Þ ðC:121Þ ðC:122Þ ðC:123Þ ðC:124Þ ðC:125Þ ðC:126Þ ðC:127Þ ðC:128Þ ðC:129Þ Appendix D Summary of Probability Theory 1. Probability distribution function: FXðxÞ ¼ PðX Zx xÞ ¼ fXðuÞ du À1 2. Probability density function: fX ðxÞ ¼ dFX ðxÞ dx 3. Gaussian probability density function: " # fX ðxÞ ¼ p1ffiffiffiffiffi  2p exp À ðx À XÞ2 22 4. Independence: fX;Y ðx; yÞ ¼ fXðxÞ fY ðyÞ 5. Expectation: Z1 E½XŠ ¼ xfXðxÞ dx À1 Z1 E½gðXފ ¼ gðxÞfXðxÞ dx ! special case : À1 Z1 E½X2Š ¼ x2fXðxÞ dx À1 Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 240 APPENDIX D: SUMMARY OF PROBABILITY THEORY 6. Variance: 2X ¼4 E½ðX À E½XŠÞ2Š ¼ E½X2Š À E2½XŠ 7. Addition of random variables: E½X þ YŠ ¼ E½XŠ þ E½YŠ 8. Independent random variables: 9. Correlation: E½X YŠ ¼ E½XŠ E½YŠ Z1Z RXY ¼ E½XYŠ ¼ xy fX;Y ðx; yÞ dx dy À1 Appendix E Definition of a Few Special Functions 1. Unit-step function (Figure E.1):  uðxÞ ¼4 1; x ! 0 0; x < 0 2. Signum function (Figure E.2):  sgnðxÞ ¼4 1; x ! 0 À1; x < 0 u(x ) 1 0 x Figure E.1 sgn(x ) 1 0 x −1 3. Rectangular function (Figure E.3): ( rectðxÞ ¼4 1; jxj 1 2 0; jxj > 1 2 Figure E.2 rect(x ) 1 −1/2 0 1/2 x Figure E.3 Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 242 APPENDIX E: DEFINITION OF A FEW SPECIAL FUNCTIONS 4. Triangular function (Figure E.4): tri(x )  triðxÞ ¼4 1 À jxj; jxj 1 0; jxj > 1 1 5. Sinc function (Figure E.5): sincðxÞ ¼4 sin x x −1 0 1 x Figure E.4 sinc(x) 1 −3 π −2 π −π 0 π 2π 3π x Figure E.5 6. Delta function (Figure E.6): Zb a f ðxÞðx À x0Þ dx ¼4  f ðx0Þ; 0; a x0 < b elsewhere δ(x ) 0 x Figure E.6 Appendix F The Q(.) and erfc Functions The Q function is defined as QðxÞ ¼4 Z p1ffiffiffiffiffi 1  exp À y2 dy 2p x 2 ðF:1Þ The function is used to evaluate the error probability of transmission systems that are disturbed by additive Gaussian noise. Some textbooks use a different function for that purpose, namely the complementary error function, abbreviated as erfc. This latter function is defined as erfcðxÞ ¼4 1 À erfðxÞ ¼ p2ffiffiffi p Z1 x expðÀy2Þ dy ðF:2Þ From Equations (F.1) and (F.2) it follows that the Q function is related to the erfc function as follows:  QðxÞ ¼ 1 erfc pxffiffi 2 2 ðF:3Þ The integral in these equations cannot be solved analytically. A simple and accurate expression (error less than 0.27 %) is given by " # QðxÞ % 1 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi expðpÀffiffixffiffi2ffi =2Þ ð1 À 0:339Þx þ 0:339 x2 þ 5:510 2p ðF:4Þ Most modern mathematical software packages such as Matlab, Maple and Mathematica comprise the erfc function as a standard function. Both functions are presented graphically in Figures F.1 and F.2. Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 244 APPENDIX F: THE Q(.) AND ERFC FUNCTIONS 100 Q(x) 10−6 Q(x) 10−1 10−7 10−2 10−8 10−3 10−9 10−4 10−10 10−5 10−11 10−6 0 1 2 3 4 5 x 10−12 3 4 5 6 7 8 x Figure F.1 100 erfc(x) 10−6 erfc(x) 10−1 10−7 10−2 10−8 10−3 10−9 10−4 10−10 10−5 10−11 10−6 0 1 2 3 4 5 x 10−12 3 4 5 6 7 8 x Figure F.2 Appendix G Fourier Transforms Definition: Xð!Þ ¼ Z1 À1 xðtÞ expðÀj!tÞ dt () xðtÞ ¼ 1Z1 2p À1 Xð!Þ expðj!tÞ d! Properties: Time domain 1: ax1ðtÞ þ bx2ðtÞ 2: xðatÞ 3: xðÀtÞ 4: xðt À t0Þ 5: xðtÞ expðj!0tÞ 6: dnxðtÞ dtn 7: Rt À1 xð Þ d 8: R1 À1 xðtÞ dt ¼! 9: xð0Þ ¼ ! 10: ðÀjtÞnxðtÞ 11: 12: xRÃ1ðtÞ À1 x1ð Þx2ðt À Þ d ðconv:Þ 13: x1ðtÞx2ðtÞ 14: R1 À1 jxðtÞj2 dt ¼! 15: XðtÞ Frequency domain aX1ð!Þ þ bX2ð!Þ 1 À!Á jaj X a XÃð!Þ Xð!Þ expðÀj!t0Þ Xð! À !0Þ ðj!ÞnXð!Þ Xð!Þ j! þ pXð0Þð!Þ ¼ ¼ X1 ðR01Þ 2p À1 Xð!Þ d! dnXð!Þ d!n XÃðÀ!Þ X1ð!Þ X2ð!Þ 1 2p R1 À1 X1ðÞX2ð! À Þ d ðconv:Þ ¼ 1 2p R1 À1 jXð!Þj2 d! ðParsevalÞ 2xðÀ!ÞðdualityÞ Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 246 APPENDIX G: FOURIER TRANSFORMS Fourier table with , , , !0 and W real constants: xðtÞ X(!) 1: ðtÞ 2: 2p 3: uðtÞ 4: 1 2 ðtÞ À 1 j2pt 5: rectðt=Þ 6: ðW=pÞ sincðWtÞ 7: triðt=Þ 8: ðW=pÞsinc2ðWtÞ 9: sgnðtÞ 10: À1 jpt 11: expðj!0tÞ 12: ðt À Þ 13: cosð!0tÞ 14: sinð!0tÞ 15: uðtÞ cosð!0tÞ 16: uðtÞ sinð!0tÞ 17: uðtÞ expðÀ tÞ 18: uðtÞt expðÀ tÞ 19: uðtÞt2 expðÀ tÞ 20: uðtÞt3 expðÀ tÞ 21: expðÀ jtjÞ 22: p1ffiffiffiffiffi  2p Àt2 exp 22  23: P1 n¼À1 ðt À nT Þ ð!Þ pð!Þ þ 1 j! uð!Þ  sincð!!=2Þ rect 2W  sinc2ð!=2Þ ! tri 2W 2 j! sgnð!Þ 2pð! À !0Þ expðÀj! Þ p½ð! À !0Þ þ ð! þ !0ފ Àjp½ð! À !0Þ À ð! þ !0ފ p 2 ½ð! À !0 Þ þ ð! þ !0ފ þ !20 j! À !2 Àj p 2 ½ð! À !0 Þ À ð! þ !0ފ þ !20 !0 À !2 1 þ j! 1 ð þ j!Þ2 2 ð þ j!Þ3 6 ð þ j!Þ4 2 2 þÀ!22!2 exp 2 2p X 1  ! À n 2p T n¼À1 T Condition >0 W >0 >0 W >0 >0 >0 >0 >0 >0 >0 Appendix H Mathematical and Physical Constants Base of natural logarithm: Logarithm of 2 to base 10: Pi: Boltzmann’s constant: Planck’s constant: Temperature in kelvin: Standard ambient temperature: Thermal energy kT at standard ambient temperature: e ¼ 2:718 281 8 logð2Þ ¼ 0:301 030 0 p ¼ 3:141 592 7 k ¼ 1:38  10À23 ½J=KŠ h ¼ 6:63  10À34 ½J sŠ Temperature in C þ 273 T0 ¼ 290 ½KŠ ¼ 17 ½CŠ kT0 ¼ 4:00  10À21 ½JŠ Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd Index Symbols -function, 5, 67, 78, 81, 83, 205, 242 discrete-time -, 82 -pulse, 51 a.c. component, 27 aliasing, 86 distortion, 52 amplifier front-end -, 146 low-noise -, 146 optical -, 145 amplitude modulated (AM) signal, 101 amplitude shift keying (ASK), 112, 173, 190 analog-to-digital conversion, 54 analytic function, 180, 181 signal, 102 antenna noise, 144 anti-causal function, 180, 181 antipodal signals, 222 attenuation, 229 autocorrelation function, 11, 69 properties of -, 13 autocorrelation of discrete-time process, 32, 57, 90 autocovariance of discrete-time process, 32 available power gain, 138 spectral density, 134 spectral density in amplifiers, 138 average time -, 14 band-limited process definition of -, 74 band-limiting filter definition of -, 74 bandpass filter, 73 signal, 101 bandpass filtering of white noise, 110 bandpass process, 44 conversion to baseband, 119 definition of -, 74 direct sampling of -, 119 properties of -, 107 bandwidth of bandpass process, 44 of low pass process, 43 root-mean-squared (r.m.s.) -, 43 baseband process definition of -, 74 Bayes’ theorem, 155 Bessel function, 113 biorthogonal signals, 225 binary detection in Gaussian noise, 158 binary phase shift keying (BPSK), 191 bipolar nonreturn-to-zero (NRZ) signal, 98 bit error probability, 155, 157 Boltzmann’s constant, 130, 247 Brownian motion, 130, 133 Butterworth filter, 98 Campbell’s theorem, 201 extension of -, 204, 206 Introduction to Random Signals and Noise W. van Etten # 2005 John Wiley & Sons, Ltd 250 INDEX CATV networks, 225 causality, 68 chain rule, 232 characteristic frequency, 104, 112 characteristic function, 194, 204 joint -, 202 of shot noise, 201 second -, 194 second joint -, 202, 204 clutter, 207 code division multiple access (CDMA), 49 coloured noise, 130 complementary error-function, 158, 243 complete orthonormal set, 216 complex envelope, 102, 105, 111 spectrum of -, 105, 111 complex impulse response, 105 complex processes, 30, 111 applied to current, 31 applied to voltage, 31 conditional probabilities, 154, 155, 159, 163 constant resistance network, 151 constants mathematical -, 247 physical -, 247 convolution, 67, 68, 160, 245 discrete -, 82, 90 correlation, 240 coefficient, 28 receiver, 171 correlation function, 11 measurement of -, 24 cost matrix, 165 covariance function auto-, 26 cross-, 26 cross-correlation, 220, 222 of discrete-time process, 32, 90 cross-correlation function, 19, 70 properties of -, 20 time averaged -, 21 cross-covariance of discrete-time process, 32 cross-power spectrum, 45 properties of -, 46 cumulant, 197 cumulative probability distribution function, 9 cyclo-stationary process, 16, 77 data signal, 19, 77, 154, 157 autocorrelation function of -, 78 spectrum of -, 78, 79 d.c. component, 27, 203 de-emphasis, 97 decibel, 118, 145, 229 list of - values, 230 decision criterion, 165 regions, 155, 156 statistics, 162 threshold, 155, 160 decision rule Bayes -, 165 maximum a posteriori (MAP) -, 165 maximum likelihood (ML) -, 165 minimax -, 165 Neyman-Pearson -, 165 demodulation coherent -, 48 synchronous -, 48, 118 derivatives, 232 detection of binary signals, 154 optimum -, 162 differentiating network, 96 differentiation, 232 digital signal processor (DSP), 50, 184 discrete Fourier transform (DFT), 84 discrete-time process, 5, 54, 193 signal, 82, 86 system, 82, 86 discrete-time Fourier transform (DTFT), 83 discriminator, 103 distance measurement, 21 metric, 162 distortionless transmission, 121 distribution function, 9 diversity, 188 doubly stochastic process, 205 duality, 245 dynamic range, 55 effective noise temperature, 139 eigenfunctions, 67 electrons, 193, 198, 207 ensemble, 3 mean, 3 envelope, 102 detection, 106, 111 distribution, 113 equalization, 99 equivalent noise bandwidth, 75, 140 noise resistance, 134 noise temperature, 134, 138 equivalent baseband system, 105 transfer function, 104 Erbium doped fiber amplifier (EDFA), 145 ergodic jointly -, 21 ergodic process definition of -, 14 ergodicity of discrete-time process, 33 error probability, 155, 164 estimation error, 176, 178 excess noise factor, 206 expansion McLaurin -, 237 Taylor -, 237 expectation, 239 false alarm, 165 fast Fourier transform (FFT), 84 filter non-recursive -, 88 recursive -, 89 tapped delay line -, 88 transversal -, 88 filtering of processes, 68 of thermal noise, 135 finite impulse response (FIR) filter, 82, 89 FM detection, 97 signal, 101 Fourier series, 57, 218 Fourier transform, 39, 57, 67, 71, 194 discrete -, 82, 84 properties of -, 245 table, 246 two-dimensional -, 202 frequency conversion, 49 frequency shift keying (FSK), 112, 161, 190 Friis’ formulas, 143 front-end amplifier, 146 Gaussian noise, 158, 162, 163, 167, 219 Gaussian processes, 27, 72 bandpass -, 112 INDEX 251 jointly -, 27 properties of -, 29 Gaussian random variables, 27 covariance matrix of -, 28 jointly -, 28 properties of -, 29 Gram-Schmidt orthogonalization, 218 group delay, 122 guard band, 62 harmonic signal, 218 Hermitian spectrum, 104 hypothesis testing, 154, 161 impulse response, 67, 68, 71, 87, 199, 201 complex -, 105 of matched filter, 161 of optimum filter, 167 independence, 239 independent processes, 10, 21 infinite impulse response (IIR) filter, 82, 89 information signal, 1, 205 inner product of signals, 216 of vectors, 215 integrals definite -, 236 indefinite -, 233 integrate-and-dump receiver, 175 interarrival time, 197 interpolation by sinc function, 52 intersymbol interference, 99 inverse discrete Fourier transform (IDFT), 84 inverse discrete-time Fourier transform (IDTFT), 83 inverse fast Fourier transform (IFFT), 84 irrelevant noise, 160, 221 jitter, 175 Kronecker delta, 216 Laplace transform, 179 light emitting diode (LED), 50 likelihood ratio, 155 linear time-invariant (LTI) filter, 65, 160, 201 system, 66 logarithms, 238 Lorentz profile, 42 low-noise amplifier, 146 252 INDEX M-ary biorthogonal signals, 225 M-ary detection in Gaussian noise, 161 M-ary phase modulation, 224 Manchester code, 98 matched filter, 161, 162, 167 for coloured noise, 167 matched impedances, 138 matching network, 149 mathematical constants, 247 relations, 231 maximum ratio combining, 189 McLaurin expansion, 237 mean ensemble -, 3 frequency, 44 of discrete-time process, 31 value, 10 mean-squared error, 52, 178 minimization of -, 176 miss, 165 mixer, 117 modems cable -, 225 telephone -, 225 modulation, 47 amplitude -, 101 by random carrier, 49 frequency -, 101 phase -, 102 moment generating function, 196 moments of random variable, 196 multiamplitude signals, 224 multiphase signals, 224 multiple-input multiple-output (MIMO) systems, 65 narrowband bandpass process definition of -, 75 system definition of -, 74 neper, 229 noise, 1 coloured -, 130 Gaussian bandpass -, 111 in optical amplifiers, 145 in systems, 137 multiplicative -, 201 presentation in signal space, 219 thermal -, 130 vector, 159, 219 noise bandwidth equivalent -, 76 noise figure average -, 140 definition of -, 140 of a cable, 141 of an attenuator, 141 of cascade, 143 spot -, 140 standard -, 140 noise in amplifiers, 138 noise temperature effective -, 139 of cascade, 143 system -, 143 noisy amplifier model, 139 nonreturn-to-zero (NRZ) signal bipolar -, 98 polar -, 79, 98 norm of a vector, 215 Norton equivalent circuit, 131 Nyquist frequency, 51 Nyquist’s theorem, 136 optical amplifier Erbium doped fiber -, 145 semiconductor -, 145 optical signal detection, 207 optimum filter characteristic, 177 optimum smoothing filter, 178 orthogonal processes, 21 quadrature processes, 109 random variables, 109 vectors, 215 orthogonal signals, 223 M-ary -, 225 orthonormal set, 158, 216 complete -, 216 orthonormal signals, 216 oscillator spectrum, 42 Parseval’s formula, 168, 245 periodically stationary process, 16 phase delay, 122 distribution, 5, 114, 115 shift, 229 phase modulation M-ary -, 224 phase reversal keying (PRK), 191 phase shift keying (PSK), 190 phasor, 102 photodetector, 198, 205 photodiode, 193, 194 avalanche -, 194, 207 photomultiplier tube, 194, 207 photons, 193, 198, 205 physical constants, 247 interpretation, 27 Planck’s constant, 130 Poisson distribution, 193 impulse process, 194, 205 processes, 193 sum formula, 81 Poisson processes homogeneous -, 193, 198 inhomogeneous -, 194, 204 sum of independent -, 196 polar nonreturn-to-zero (NRZ) signal, 79, 98 posterior probabilities, 165 power, 40 a.c. -, 27 electrical -, 133 in output process, 71 maximum transfer of -, 138, 147 of discrete-time process, 57 of stochastic process, 133 power spectrum, 39 cross-, 45 measurement of -, 116 properties of -, 40 pre-emphasis, 97 pre-envelope, 102 prediction, 175, 179 discrete-time -, 184 pure -, 184 pure -, 179 prior probabilities, 154, 163, 165 probability density function, 10, 239 Gaussian -, 27, 239 joint -, 10 Laplacian -, 186 Poisson -, 193 INDEX 253 probability distribution function, 239 joint Nth-order -, 10 second-order -, 10 process bandpass -, 48, 106 stationary Nth-order -, 11 first-order -, 10 second-order -, 11 Q-function, 158, 164, 243 quadrature components, 102, 106, 118 measurement of -, 118 description of bandpass processes, 106 description of modulated signals, 101 processes, 107–109 signals, 218 quadrature amplitude modulated (QAM) signals, 225 quadrature phase shift keying (QPSK), 224 quantization, 54 characteristic, 55 error, 55 levels, 55 noise, 56 step size, 55 quantizer, 55 non-uniform -, 57 uniform -, 56 queuing, 197 radar, 207 detection, 165 ranging, 22 random gain, 205 signal, 1 variable, 2 vector, 155 random data signal, 77, 154 spectrum of -, 78 random point processes, 193 random sequence continuous -, 5 discrete -, 7 random-pulse process, 205 Rayleigh-distribution, 113 RC-network, 71, 76, 137 realization, 3 254 INDEX reconstruction of bandpass processes, 119 of sampled processes, 52 of sampled signals, 51 rectangular function, 241 pulse, 79, 84, 201 rectifier, 103 relevant noise, 159, 221 return-to-zero signal unipolar -, 80 Rice-distribution, 113 root-mean-squared (r.m.s.) bandwidth, 43 value, 27 sample function, 3 sampling, 5 direct -, 119 flat-top -, 62 ideal -, 51 of bandpass processes, 119 rate, 51, 52, 119 theorem for deterministic signals, 51 for stochastic processes, 52 Schwarz’s inequality, 166, 216 semi-invariant, 197 semiconductor optical amplifier (SOA), 145 series, 237 shot noise, 199 signal constellation, 219, 222 energy, 159, 162, 218 harmonic -, 218 restoration, 179 space, 161, 163, 216 vector, 159, 216 signal-to-noise ratio, 140, 164, 166 matched filter output -, 168 signal-to-quantization-noise ratio, 56 signal-to-shot-noise ratio, 201, 206 signum function, 241 simplex signal set, 163, 164, 227 sinc function, 52, 81, 84, 242 single-input single-output (SISO) systems, 65 smoothing, 175, 176 discrete-time -, 183 spectrum analyzer, 116 of data signal, 77 of discrete-time process, 57 of filter output, 71 spill-over, 63, 99 split-phase code, 98 spread spectrum, 50 stable system, 87 stationary points, 233 processes, 9 stochastic processes, 2 continuous -, 4 discrete -, 4, 6 discrete-time -, 4, 5, 31 independent -, 10 spectra of -, 39 strict-sense stationary process, 11 sufficient statistic, 160 superheterodyne, 116 switching center, 208 synchronization, 175 system causal -, 179 optimal -, 153 stable -, 180 synthesis, 153 Taylor expansion, 237 The´venin equivalent circuit, 131 thermal noise, 130 current in passive networks, 137 in passive networks, 131 spectrum of current in a resistor, 131 spectrum of voltage across a resistor, 130 voltage in passive networks, 136 time average, 14 transfer function, 67, 89 equivalent baseband -, 104 of bandpass filter, 104 triangular function, 242 trigonometric relations, 231 uncorrelated processes, 27 unipolar return-to-zero, 80 unit-step function, 241 INDEX 255 variance, 26, 55, 197, 201, 204, 240 vector spaces, 215 voice channel, 62 waiting time, 197 white noise, 70, 75, 91, 129, 158, 163, 167, 203, 205, 219 bandpass filtering of -, 110 whitening filter, 169 wide-sense stationary, 219 jointly -, 175 wide-sense stationary processes, 71, 175, 206 definition of -, 12 discrete-time -, 57, 90 jointly -, 20, 70 Wiener filter, 154, 175 discrete time -, 183 Wiener-Hopf equation, 179 Wiener-Khinchin relation, 39 z-transform, 57, 86

Top_arrow
回到顶部
EEWORLD下载中心所有资源均来自网友分享,如有侵权,请发送举报邮件到客服邮箱bbs_service@eeworld.com.cn 或通过站内短信息或QQ:273568022联系管理员 高进,我们会尽快处理。