Sampling Theorem

Abstract

We enjoy the power and convenience of mobile communications today because of a very important exchange made early in the history of this technology. Once upon a time, radios were completely stunted by size, weight, and power limits. Just think of the backpack-sized radios that infantrymen carried around during World War II. The primary reason we enjoy tiny mobile radios (i.e. cellular phones) is that the analog design burdens were shifted to the digital domain which in turn placed the attendent digital algorithms onto integrated circuit technology, whose power vs. cost ratio accelerated favorably over the ensuing decades. The key bridge between the analog and the digital is sampling. In terms of basic electronics, an analog signal voltage is applied to a sample-and-hold circuit and held there by a tiny bit of capacitance while a bank of comparators marks off the proportional integer value. These values are the digital samples of the analog signal. The challenge is to analyze the original analog signal from its collected samples. The sampling theorem provides the primary mathematical mechanism for doing this.

Appendix

Listing 2.1: The first line ensures that we use floating-point division instead of the default integer divide. Line 3 establishes the figure and axis bindings using subplots. Keeping these separate is useful for very complicated plots. The arange function creates a Numpy array of numbers. Then, we compute the sine of this array and plot it in the figure we just created. The -o is shorthand for creating a plot with solid lines and with points marked with o symbols. Attaching the plot to the ax variable is the modern convention for Matplotlib that makes it clear where the plot is to be drawn. The next two lines set up the labels for the x-axis and y-axis, respectively, with the specified font size.

 1 from __future__ import division

 2

 3 fig,ax = subplots()

 4 f = 1.0  # Hz, signal frequency

 5 fs = 5.0 # Hz, sampling rate (ie. >= 2*f)

 6 t = arange(-1,1+1/fs,1/fs) # sample interval, symmetric

 7                            # for convenience later

 8 x = sin(2*pi*f*t)

 9 ax.plot(t,x,’o-’)

10 ax.set_xlabel(’Time’,fontsize=18)

11 ax.set_ylabel(’Amplitude’,fontsize=18)

Listing 2.2: Listing corresponding to Fig. 2.2. On the third line, we establish the limits of the axis using keyword arguments. This enhances clarity because the reader does not have to otherwise look up the positional arguments instead.

1 fig,ax = subplots()

2 ax.plot(t,x,’o-’)

3 ax.axis(xmin = 1/(4*f)-1/fs*3,

4         xmax = 1/(4*f)+1/fs*3,

5         ymin = 0,

6         ymax = 1.1 )

7 ax.set_xlabel(’Time’,fontsize=18)

8 ax.set_ylabel(’Amplitude’,fontsize=18)

Listing 2.3: Code for constructing the piecewise linear approximation. The hstack function packs smaller arrays horizontally into a larger array. The remainder of the code formats the respective inputs for Numpy’s piecewise linear interpolating function.

1 interval=[] # piecewise domains

2 apprx = []  # line on domains

3 # build up points *evenly* inside of intervals

4 tp = hstack([linspace(t[i],t[i+1],20,False) for i in range(len(t)-1)])

5 # construct arguments for piecewise

6 for i in range(len(t)-1):

7    interval.append(logical_and(t[i] <= tp,tp < t[i+1]))

8    apprx.append((x[i+1]-x[i])/(t[i+1]-t[i])*(tp[interval[-1]]-t[i]) + x[i])

9 x_hat = piecewise(tp,interval,apprx) # piecewise linear approximation

Listing 2.4: Listing corresponding to Fig. 2.3. Line 2 uses fill_between to fill in the convex region between the x_hat and the sin function with the given facecolor. Because we want a vertical axis on both sides of the figure, we use the twinx function to create the duplicated axis. This shows the value of keeping separate variables for axes (i.e. ax1,ax2).

 1 fig,ax1=subplots()

 2 # fill in the difference between the interpolant and the sine

 3 ax1.fill_between(tp,x_hat,sin(2*pi*f*tp),facecolor=’red’)

 4 ax1.set_xlabel(’Time’,fontsize=18)

 5 ax1.set_ylabel(’Amplitude’,fontsize=18)

 6 ax2 = ax1.twinx() # create clone of ax1

 7 sqe = (x_hat-sin(2*pi*f*tp))**2 #compute squared-error

 8 ax2.plot(tp, sqe,’r’)

 9 ax2.axis(xmin=-1,ymax= sqe.max() )

10 ax2.set_ylabel(’Squared error’, color=’r’,fontsize=18)

11 ax1.set_title(’Errors with Piecewise Linear Interpolant’,fontsize=18)

Listing 2.5: Code corresponding to Fig. 2.4 that shows that you can draw multiple lines with a single plot function. The only drawback is that you cannot later refer to the lines individually using the legend function. Note the squared-error here is imperceptible in this plot due to improved interpolant.

1 fig,ax=subplots()

2 t = linspace(-1,1,100)      # redefine this here for convenience

3 ts = arange(-1,1+1/fs,1/fs) # sample points

4 num_coeffs=len(ts)

5 sm=0

6 for k in range(-num_coeffs,num_coeffs): # since function is real, need both

7    sides sm+=sin(2*pi*(k/fs))*sinc(k - fs*t)

8 ax.plot(t,sm,’--’,t,sin(2*pi*t),ts, sin(2*pi*ts),’o’)

9 ax.set_title(’Sampling Rate=%3.2f Hz’ % fs, fontsize=18 )

Listing 2.6: Listing for Fig. 2.5. Note that on Line 8, we scale the y-axis maximum using the Numpy unary function (max()) attached to the sqe variable.

 1 fig,ax1=subplots()

 2 ax1.fill_between(t,sm,sin(2*pi*f*t),facecolor=’red’)

 3 ax1.set_ylabel(’Amplitude’,fontsize=18)

 4 ax1.set_xlabel(’Time’,fontsize=18)

 5 ax2 = ax1.twinx()

 6 sqe = (sm - sin(2*pi*f*t))**2

 7 ax2.plot(t, sqe,’r’)

 8 ax2.axis(xmin=0,ymax = sqe.max())

 9 ax2.set_ylabel(’Squared Error’, color=’r’,fontsize=18)

10 ax1.set_title(r’Errors with Whittaker Interpolant’,fontsize=18)

Listing 2.7: Listing corresponding to Fig. 2.6 introducing the annotate function that draws arrows with indicative text on the plot. The shrink key moves the tip and base of the arrow a small percentage away from the annotation point.

 1 fig,ax=subplots()

 2 k=0

 3 fs=2 # makes this plot easier to read

 4 ax.plot(t,sinc(k - fs * t),

 5         t,sinc(k+1 - fs * t),’--’,k/fs,1,’o’,(k)/fs,0,’o’,

 6         t,sinc(k-1 - fs * t),’--’,k/fs,1,’o’,(-k)/fs,0,’o’

 7 )

 8 ax.hlines(0,-1,1)  # horizontal lines

 9 ax.vlines(0,-.2,1) # vertical lines

10 ax.annotate(’sample value goes here’,

11             xy=(0,1),          # arrowhead position

12             xytext=(-1+.1,1.1),# text position

13             arrowprops={’facecolor’:’red’,

14                            ’shrink’:0.05},

15             )

16 ax.annotate(’no interference here’,

17             xy=(0,0),

18             xytext=(-1+.1,0.5),

19             arrowprops={’facecolor’:’green’,’shrink’:0.05},

20             )

Listing 2.8: Listing for Fig. 2.7. Line 4 uses Numpy broadcasting to create an implicit grid for evaluating the interpolating functions. The .T suffix is the transpose. The sum(axis=0) is the sum over the rows.

 1 fs=5.0 # sampling rate

 2 k=array(sorted(set((t*fs).astype(int)))) # sorted coefficient list

 3 fig,ax = subplots()

 4 ax.plot(t,(sin(2*pi*(k[:,None]/fs))*sinc(k[:,None]-fs*t)).T,’--’, # individual

 5         whittaker functions t,(sin(2*pi*(k[:,None]/fs))*sinc(k[:,None]-fs*t)).

 6         sum(axis=0),’k-’, # whittaker interpolant k/fs,

 7         sin(2*pi*k/fs),’ob’)# samples

 8 ax.set_xlabel(’Time’,fontsize=18)

 9 ax.set_ylabel(’Amplitude’,fontsize=18)

10 ax.set_title(’Sine Reconstructed with Whittaker Interpolator’)

11 ax.axis((-1.1,1.1,-1.1,1.1));

Listing 2.9: Listing corresponding to Fig. 2.8.

 1 t = linspace(-5,5,300) # redefine this here for convenience

 2 fig,ax = subplots()

 3

 4 fs=5.0     # sampling rate

 5 ax.plot(t,sinc(fs * t))

 6 ax.grid()  # put grid on axes

 7 ax.annotate(’This keeps going...’,

 8             xy=(-4,0),

 9             xytext=(-5+.1,0.5),

10             arrowprops={’facecolor’:’green’,

11                            ’shrink’:0.05},

12             fontsize=14)

13 ax.annotate(’... and going...’,

14             xy=(4,0),

15             xytext=(3+.1,0.5),

16             arrowprops={’facecolor’:’green’,

17                            ’shrink’:0.05},

18             fontsize=14)

Listing 2.10: Listing corresponding to Fig. 2.9. The eigvalsh function comes from the LAPACK compiled library.

 1 def kernel(x,sigma=1):

 2     ’convenient function to compute kernel of eigenvalue problem’

 3     x = np.asanyarray(x)           # ensure x is array

 4     y = pi*where(x == 0,1.0e-20, x)# avoid divide by zero

 5     return sin(sigma/2*y)/y

 6

 7 nstep=100                # quick and dirty integral quantization

 8 t = linspace(-1,1,nstep) # quantization of time

 9 dt = diff(t)[0]          # differential step size

10 def eigv(sigma):

11     return eigvalsh(kernel(t-t[:,None],sigma)).max() # compute max eigenvalue

12

13 sigma = linspace(0.01,4,15) # range of time-bandwidth products to consider

14

15 fig,ax = subplots()

16 ax.plot(sigma, dt*array([eigv(i) for i in sigma]),’-o’)

17 ax.set_xlabel(’Time-bandwidth product $\sigma$’,fontsize=18)

18 ax.set_ylabel(’Maximum Eigenvalue’,fontsize=18)

19 ax.axis(ymax=1.01)

20 ax.grid()

Listing 2.11: Listing corresponding to Fig. 2.10. The argmax function finds the array index corresponding to the array maximum.

1 sigma=3  # time-bandwidth product

2 w,v=eigh(kernel(t-t[:,None],sigma)) # eigen-system

3 maxv=v[:, w.argmax()]               # eigenfunction for max eigenvalue

4 fig,ax=subplots()

5 ax.plot(t,maxv)

6 ax.set_xlabel(’$t$’,fontsize=22)

7 ax.set_ylabel(’$\psi_0(t)$’,fontsize=22)

8 ax.set_title(’Eigenvalue=%3.4f;$\sigma$=%3.2f’%(w.max()*dt,sigma))

Listing 2.12: Listing corresponding to Fig. 2.11. the sign function computes the sign of the argument.

 1 def kernel_tau(x,W=1):

 2     ’convenient function to compute kernel of eigenvalue problem’

 3     x = np.asanyarray(x)

 4     y = pi*where(x == 0,1.0e-20, x) # avoid divide by zero

 5     return sin(2*W*y)/y

 6

 7 nstep=300                # quick and dirty integral quantization

 8 t = linspace(-1,1,nstep) # quantization of time

 9 tt = linspace(-2,2,nstep)# extend interval

10 sigma = 5

11 W = sigma/2./2./t.max()

12 w,v=eig(kernel_tau(t-tt[:,None],W)) # compute e-vectors/e-values

13 maxv=v[:,w.real.argmax()].real      # take real part

14

15 fig,ax = subplots()

16 ax.plot(tt,maxv/sign(maxv[nstep/2])) # normalize for orientation

17 ax.set_xlabel(’$t$’,fontsize=24)

18 ax.set_ylabel(r’$\phi_{max}(t)$’,fontsize=24)

19 ax.set_title(’$\sigma=%d$’%(2*W*2*t.max()),fontsize=26)

Listing 2.13: Setup code for gyphs. The add_patch function places graphics primitives (“patches”) like rectangles and arrows onto the plot. The FancyArrow is one of the graphics primitives for an arrow. The gridspec is a more powerful tool than subplots for controlling placement of plots on a grid.

 1 def facet_filled(x,alpha=0.5,color=’b’):

 2     ’construct 3D facet from adjacent points filled to zero’

 3     a,b=x

 4     a0= a*array([1,1,0])

 5     b0= b*array([1,1,0])

 6     ve = vstack([a,a0,b0,b])      # create closed polygon facet

 7     poly = Poly3DCollection([ve]) # create facet

 8     poly.set_alpha(alpha)         # set transparency

 9     poly.set_color(color)

10     return poly

11

12 def drawDFTView(X,ax=None,fig=None):

13     ’Draws 3D diagram given DFT matrix’

14     a=2*pi/len(X)*arange(len(X))

15     d=vstack([cos(a),sin(a),array(abs(X)).flatten()]).T

16     if ax is None and fig is None:

17         fig = plt.figure()

18         fig.set_size_inches(6,6)

19

20     if ax is None: # add ax to existing figure

21         ax = fig.add_subplot(1, 1, 1, projection=’3d’)

22

23     ax.axis([-1,1,-1,1])          # x-y limits

24     ax.set_zlim([0,d[:,2].max()]) # z-limit

25     ax.set_aspect(1)              # aspect ratio

26     ax.view_init(azim=-30)        # camera view position

27     a=FancyArrow(0,0,1,0,width=0.02,length_includes_head=True)

28     ax.add_patch(a)

29     b=FancyArrow(0,0,0,1,width=0.02,length_includes_head=True)

30     ax.add_patch(b)

31     art3d.patch_2d_to_3d(a) # format 2D patch for 3D plot

32     art3d.patch_2d_to_3d(b)

33     ax.axis(’off’)

34

35     sl=[slice(i,i+2) for i in range(d.shape[0]-2)] # collect neighboring points

36     for s in sl:

37       poly=facet_filled(d[s,:])

38       ax.add_collection3d(poly)

39

40     # edge polygons

41     ax.add_collection3d(facet_filled(d[[-1,0],:]))

42     ax.add_collection3d(facet_filled(d[[-2,-1],:]))

43 def drawInOut(X,v,return_axes=False):

44     fig = plt.figure()

45     fig.set_size_inches(8,8)

46     gs = gridspec.GridSpec(8,6)

47

48     ax1 = plt.subplot(gs[3:5,:2])

49     ax2 = plt.subplot(gs[:,2:],projection=’3d’)

50

51

52     ax1.stem(arange(len(v)),v)

53     ymin,ymax= ax1.get_ylim()

54     ax1.set_ylim(ymax = ymax*1.2, ymin = ymin*1.2)

55     ax1.set_title(’Signal’)

56     ax1.set_xlabel(’n’)

57     ax1.tick_params(labelsize=8)

58

59     drawDFTView(X,ax2)

60     if return_axes:

61         return ax1,ax2