Every transform we have used so far — DTFT, $z$-transform, DFT — analyzes a signal as a whole. The basis functions $e^{j\omega n}$ extend over all time, so a single Fourier coefficient mixes together contributions from the entire signal. That is perfectly adequate for stationary signals（平穩訊號）whose spectral content does not change, but most signals we actually care about — speech, music, radar returns, biomedical signals — are nonstationary（非平穩訊號）: their frequency content evolves with time. A melody is precisely a sequence of different frequencies at different times.
The lecture opens with the Fourier transform of an amplitude-modulated sine: the FT shows which frequencies are present, but tells us almost nothing about when they occur. Here is the cleanest demonstration. Take two signals built from the same two tones, played in opposite order:
The fix suggested by the slides is intuitive: if the spectrum changes with time, chop the signal into short segments and Fourier-transform each segment separately. Within a segment of a few tens of milliseconds, speech or music is approximately stationary（短時間內近似平穩）, so a per-segment spectrum is meaningful. Doing this for every segment position, and stacking the resulting spectra side by side as columns of an image, is exactly the idea of the spectrogram （頻譜圖／聲譜圖）, and the underlying transform is the Short-Time Fourier Transform (STFT, 短時傅立葉轉換).
Let $w[n]$ be a window function（窗函數）of length $L$ — real, nonnegative, concentrated around its center, and zero outside $0\le n\le L-1$. The STFT of $x[n]$ is the DTFT of the windowed signal, with the window slid to time position $n$:
For each fixed window position $n$, $X(n,\omega)$ is an ordinary DTFT of the segment $x[m]\,w[m-n]$ — a function of continuous frequency $\omega$, periodic in $2\pi$. As $n$ varies, the window slides along the signal, so the STFT is a two-dimensional function: one time axis, one frequency axis.
The STFT can be grouped two ways, and both matter:
In practice we evaluate the STFT only on a grid: time is sampled every $M$ samples (the hop size, 跳距), and frequency is sampled at the $N$ DFT bins. With frame index $m$ and bin index $k$:
In words: take the $N$ samples starting at $mM$, multiply pointwise by the window, and take an $N$-point FFT. Each $m$ produces one frame（音框）— one column of the time–frequency image. For a signal of length $L_x$, the number of complete frames is
which is precisely the loop bound used in the homework's myspectrogram
function. The frequency spacing between adjacent bins is $f_s/N$ — but as we will see in
Section 9.3.2, the true resolution is set by the window length, not by $N$.
Two elementary inputs reveal exactly what the STFT can and cannot resolve:
So the window $w[n]$ acts as the "point spread function" of the analysis in both axes simultaneously. Choosing $w$ is the entire art of the STFT — which is why the lecture spends four slides on it.
Truncating a signal with a rectangular window（矩形窗）is multiplication in time, hence convolution with the window spectrum in frequency. The rectangular window's spectrum is the periodic sinc (Dirichlet kernel): a narrow mainlobe, but sidelobes（旁瓣） only $-13$ dB down. Those sidelobes cause leakage（頻譜洩漏）: a strong tone splatters energy across the whole band and buries weak neighbors. Tapered windows trade a wider mainlobe for far lower sidelobes.
| Window | Mainlobe width (null–null) | Peak sidelobe | Typical use |
|---|---|---|---|
| Rectangular（矩形） | $4\pi/L$ | $-13$ dB | maximum resolution, transient analysis |
| Hann（漢恩） | $8\pi/L$ | $-31$ dB | default for spectrograms / STFT |
| Hamming（漢明） | $8\pi/L$ | $-41$ dB | speech analysis |
| Blackman（布雷克曼） | $12\pi/L$ | $-57$ dB | large dynamic range |
dsp10.m and virtually
every audio tool default to it.
Window shape sets the sidelobe behavior; window length $L$ sets the fundamental trade-off. The mainlobe width scales as $1/L$, so in hertz the frequency resolution of a Hann-windowed STFT is approximately
You cannot win on both axes at once. This is the signal-processing form of the uncertainty principle（測不準原理）:
with equality only for the Gaussian window. Improving time resolution by shortening the window necessarily degrades frequency resolution in exact proportion, and vice versa.
A useful mental picture: each STFT coefficient $X_m[k]$ "owns" a cell of the time–frequency plane of width $\Delta t$ and height $\Delta f$, with area bounded below by the uncertainty principle. The window length only decides the aspect ratio of the cells — never their area.
The spectrogram（頻譜圖）is the squared magnitude of the STFT, almost always displayed in decibels as an image: horizontal axis time, vertical axis frequency, color/brightness $=$ energy. It discards the STFT phase and keeps the part the eye can read.
Axis calibration for sampling rate $f_s$, hop $M$, DFT length $N$ (exactly the axes
constructed in dsp10.m):
For real signals only bins $0\le k< N/2$ are kept ($f$ up to $f_s/2$); the upper half is the mirror image. The dB scale matters: audio features of interest (harmonics, formants, reverb tails) span 60–80 dB of dynamic range and would be invisible on a linear scale.
The classic test signal for time–frequency analysis is the linear chirp （線性掃頻訊號）, whose instantaneous frequency rises linearly:
stft, 87.5% overlap, 60 dB display range). Left, $L=64$ (32 ms): the ridge is thick in frequency but time events would be sharp — wideband. Right, $L=1024$ (512 ms): the ridge is spectrally thin, but the line blurs along time and bends at the edges because the chirp sweeps ~77 Hz within each window — narrowband.（同一訊號、同一不確定性原理：短窗時間準、頻率糊；長窗頻率準、時間糊。）The lecture's slide "Narrowband vs. Wideband" summarizes the effect of varying window length on real sounds (speech is the standard example):
| Narrowband spectrogram（窄帶） | Wideband spectrogram（寬帶） | |
|---|---|---|
| Window length | long (e.g. > 20 ms for speech) | short (e.g. < 5 ms) |
| Filter bandwidth | narrow → fine $\Delta f$ | wide → fine $\Delta t$ |
| What you see | individual pitch harmonics as horizontal lines（諧波線） | individual glottal pulses as vertical striations（聲門脈衝直紋） |
| What blurs | onsets, plosives, fast transitions | closely spaced harmonics merge; formant envelopes（共振峰）stand out |
The hop size $M$ (frame advance) controls how densely the time axis is sampled.
Overlap（重疊）is $L-M$ samples, i.e. overlap fraction $1-M/L$. Typical
choices are 50% ($M=L/2$) or 75% ($M=L/4$ — the choice in dsp10.m, where
$M=N/4$).
MATLAB's Signal Processing Toolbox provides spectrogram directly; the
homework instead builds it from first principles (frame → window → FFT → magnitude
squared). Both in one place:
% Toolbox one-liner: Hann window, length N, 75% overlap, N-point DFT
N = 1024; M = N/4;
spectrogram(x, hann(N), N-M, N, Fs, 'yaxis'); % draws dB image directly
% Equivalent from scratch (the dsp10.m approach)
w = 0.5*(1 - cos(2*pi*(0:N-1)'/N)); % Hann window
num_frames = floor((length(x)-N)/M) + 1;
X = zeros(N, num_frames);
for m = 1:num_frames
seg = x((m-1)*M+1 : (m-1)*M+N) .* w; % frame x window
X(:, m) = abs(fft(seg)).^2; % power spectrum
end
t = (0:num_frames-1)*M/Fs; f = (0:N/2-1)*Fs/N;
image(t, f, 10*log10(X(1:N/2,:))); axis xy; colormap(hot(256)); colorbar;
spectrogram(x, window, noverlap, nfft, fs) takes the overlap
(samples shared between consecutive frames), not the hop size. For hop
$M$ pass noverlap = N - M. Passing the hop by mistake silently produces a
spectrogram with the wrong time density. Also remember axis xy when using
image — otherwise the frequency axis is upside-down.
Fix one frequency $\omega_k$ and watch $X(n,\omega_k)$ evolve with $n$. Rearranging the STFT definition (with the window treated as a filter impulse response):
So each STFT frequency row is produced by: (1) heterodyne（混頻）— multiply by $e^{-j\omega_k n}$ to shift the band around $\omega_k$ down to DC; (2) lowpass filter with $\tilde w[n]$, whose bandwidth is the window mainlobe; (3) keep every $M$-th output sample (the hop). The STFT is therefore a uniform filterbank（濾波器組）of $N$ bandpass channels, each of bandwidth $\approx c\,f_s/L$, spaced $f_s/N$ apart, downsampled by $M$.
This view explains the vocabulary of Section 9.4.3: a long window is a narrow lowpass filter — hence "narrowband" spectrogram — and a short window is a wide one. It also connects the STFT to Chapter 8: a DFT of one windowed frame is just this filterbank evaluated at a single time instant.
The STFT is not just for pictures — because it is invertible (when sampled densely enough), we can modify a signal in the time–frequency domain and resynthesize: noise suppression, time stretching, pitch shifting, equalization. The standard synthesis is overlap-add (OLA, 重疊相加): inverse-FFT each (possibly modified) frame, then add the frames back at their original positions:
If the frames are unmodified, each IFFT returns exactly $x[n+mM]\,w[n]$, so the OLA sum gives $y[n]=x[n]\sum_m w[n-mM]$. Perfect reconstruction (up to a constant) therefore requires the shifted windows to add to a constant — the COLA (constant-overlap-add) condition（恆定重疊相加條件）:
For the length-$L$ periodic Hann window, COLA holds at hop $M=L/2$ (with $C=1$) and at $M=L/4$ (with $C=2$) — one reason the Hann window dominates analysis–synthesis practice. The four "STFT Analysis-Synthesis" slides walk this chain: window → FFT → modify → IFFT → overlap-add, with COLA guaranteeing transparency when nothing is modified.
$X_m[k]=\sum_{n=0}^{N-1}x[n+mM]w[n]e^{-j2\pi kn/N}$: slide a length-$L$ window by hops of $M$, FFT each frame. Time grid $mM/f_s$, frequency grid $kf_s/N$.
$S[m,k]=|X_m[k]|^2$, displayed in dB. Rows = bandpass channel outputs, columns = frame spectra. Keep $k<N/2$ for real signals.
Mainlobe ↔ resolution, sidelobes ↔ leakage. Rect: $4\pi/L$, $-13$ dB. Hann: $8\pi/L$, $-31$ dB, COLA at 50%/75% overlap — the default.
$\Delta t\approx L/f_s$, $\Delta f\approx c f_s/L$, $\Delta t\,\Delta f\ge 1/4\pi$. Long = narrowband (harmonics), short = wideband (transients). Zero-padding interpolates only.
Hop $M$, overlap $L-M$; 50–75% typical. spectrogram(x,hann(N),N-M,N,Fs) takes overlap, not hop. Frames: $\lfloor (L_x-N)/M\rfloor+1$.
IFFT frames + overlap-add; perfect reconstruction iff $\sum_m w[n-mM]=C$ (COLA). Hann at $M=L/2$: $C=1$; at $M=L/4$: $C=2$.
Problem: Let $x_1[n]$ consist of a 50 Hz tone for the first half-second followed by a 120 Hz tone for the second half-second ($f_s=1$ kHz), and let $x_2[n]$ play the same two tones in the opposite order (Fig. 9-1). (a) Explain why $|X_1(e^{j\omega})| \approx |X_2(e^{j\omega})|$. (b) Where, mathematically, does the FT store the ordering information? (c) Describe what the STFT magnitude of each signal looks like and why it distinguishes them.
(a) Write $x_2[n]=x_1[n_0-1-n]$ approximately (the second signal is a time-reversal-plus-shift of the first up to phase of the tones; more simply, each is a sum of the same two gated tones, just gated by complementary halves). A gated tone $g[n]\cos(\omega_i n)$ has spectrum $\tfrac12 G(e^{j(\omega-\omega_i)})+\tfrac12 G(e^{j(\omega+\omega_i)})$ where $G$ is the gate's spectrum. Shifting the gate in time multiplies $G$ by a pure phase $e^{-j\omega n_d}$ and leaves $|G|$ unchanged. Since both signals are sums of the same two gated tones with gates that differ only by a time shift, and the two spectral lumps (50 and 120 Hz) barely overlap, the magnitude spectra are nearly identical — only the cross terms in the overlap region differ slightly.
(b) In the phase: a delay of $n_d$ samples contributes the linear phase factor $e^{-j\omega n_d}$. The ordering of events is encoded in how the phase of each spectral lump slopes with $\omega$ (group delay $-\,d\angle X/d\omega$ evaluated near each tone gives that tone's arrival time).
(c) With a window much shorter than 0.5 s, the STFT of $x_1$ shows a ridge at 50 Hz for $t<0.5$ s that jumps to 120 Hz afterwards; $x_2$ shows the mirror pattern. The STFT separates them because each frame sees only one regime — it converts the unreadable phase code into a readable time axis.
Problem: A signal sampled at $f_s=8$ kHz contains two simultaneous tones at 1000 Hz and 1040 Hz. (a) Using a Hann window (mainlobe null-to-null width $8\pi/L$), find the minimum window length $L$ that resolves the two tones, and the corresponding window duration. (b) Would zero-padding the FFT from $N=L$ to $N=8L$ help if $L$ is below your answer? (c) For the rectangular window, what $L$ suffices, and why might you still prefer the Hann window?
(a) The Hann mainlobe null-to-null width in rad/sample is $8\pi/L$, i.e. in hertz $\Delta f = \frac{8\pi/L}{2\pi}f_s = \frac{4f_s}{L}$. Two equal tones are (just) resolved when their spacing exceeds roughly half the null-to-null width, but the safe design rule is to require the full spacing to exceed the half-width $\frac{4f_s}{L}\cdot\frac12$... To keep it simple use the standard criterion $\Delta f_{\min}\approx 4f_s/L \le 40$ Hz:
Choose the next power of two, $L=1024$, i.e. a window duration of $1024/8000=128$ ms. (Any consistent mainlobe criterion is acceptable; the scaling $L\propto f_s/\Delta f$ is the point.)
(b) No. Zero-padding evaluates the same windowed spectrum — the convolution of the true line spectrum with the same mainlobe — on a denser grid. If the two mainlobes have already merged into one bump, sampling that bump more finely cannot split it. Only more data (larger $L$) narrows the mainlobe.
(c) The rectangular mainlobe is $4\pi/L$ (half of Hann's), so $L\ge 2f_s/\Delta f = 400$ samples suffices. But its $-13$ dB sidelobes leak badly: if the two tones have unequal amplitudes (or there is any other strong component), the weaker tone can be buried under the stronger one's sidelobes. Hann's $-31$ dB sidelobes cost a factor of 2 in length but buy 18 dB of dynamic range.
Problem: A speech signal is sampled at $f_s=10$ kHz. The talker's pitch (fundamental) is $f_0=200$ Hz, so harmonics are spaced 200 Hz apart and glottal pulses occur every $1/f_0 = 5$ ms. Using the Hann resolution rule $\Delta f\approx 4f_s/L$: (a) find the window lengths (samples and ms) for a narrowband spectrogram that clearly resolves the harmonics ($\Delta f \le 100$ Hz). (b) Find the window length for a wideband spectrogram that resolves individual glottal pulses in time ($\Delta t \le 2.5$ ms). (c) Show that no single window can do both, by computing $\Delta t\,\Delta f$ demanded by (a)+(b) and comparing with the uncertainty bound.
(a) Need $4f_s/L\le 100$ Hz $\Rightarrow L\ge 4\cdot10000/100=400$ samples $=40$ ms. A 40 ms Hann window spans 8 pitch periods; harmonics appear as clean horizontal lines, but each glottal event is smeared over 40 ms — this is the narrowband setting.
(b) Time resolution is the window duration: $\Delta t\approx L/f_s\le 2.5$ ms $\Rightarrow L\le 25$ samples... in practice $L=25$ is extreme; taking $\Delta t = L/f_s = 2.5$ ms gives $L = 25$ samples, and commonly one relaxes to $L\approx 30$–$50$ ($3$–$5$ ms). Each window now spans less than one pitch period: vertical striations mark the glottal pulses — the wideband setting. Its frequency resolution is $4f_s/L\approx 1600$ Hz at $L=25$: individual harmonics merge, and only the broad formant envelope survives.
(c) Demanding both simultaneously requires $\Delta t\,\Delta f \le (2.5\ \text{ms})(100\ \text{Hz}) = 0.25$. For a single Hann window the product is fixed: $\Delta t\,\Delta f = \frac{L}{f_s}\cdot\frac{4 f_s}{L} = 4$, independent of $L$ — sixteen times larger than required, and the theoretical floor for any window is $\Delta t\,\Delta f \ge 1/(4\pi)\approx 0.08$ with the practical mainlobe-based product always of order 1. Hence one analysis cannot satisfy both specs; speech researchers simply make two spectrograms, one narrowband and one wideband.
Problem: Using the definition $X(n,\omega)=\sum_m x[m]\,w[m-n]\,e^{-j\omega m}$ with a length-$L$ window $w$ whose DTFT is $W(e^{j\omega})$, derive the STFT of (a) the complex exponential $x[m]=e^{j\omega_0 m}$ and (b) the shifted impulse $x[m]=\delta[m-m_0]$. (c) Interpret both results in terms of the spectrogram image and the tiling picture of Fig. 9-3.
(a) Substitute and change variable $r=m-n$:
Magnitude: $|X(n,\omega)|=|W(e^{j(\omega-\omega_0)})|$ for every $n$ — a horizontal ridge at $\omega_0$ whose cross-section is the window spectrum (mainlobe width sets the ridge thickness, sidelobes appear as faint parallel stripes).
(b) The sum collapses at $m=m_0$:
A vertical stripe at times $n$ near $m_0$, identical at all frequencies, whose cross-section along $n$ is the (reversed) window shape — width $=$ window length.
(c) The tone (a horizontal line in the ideal time–frequency plane) is blurred vertically by the window's frequency width $\Delta f$; the impulse (a vertical line) is blurred horizontally by the window's duration $\Delta t$. Every signal feature is smeared by exactly one tile of Fig. 9-3 — the STFT renders the time–frequency plane at the resolution of its window, no finer.
Problem: An audio clip has $L_x=480{,}000$ samples ($f_s=48$ kHz, 10 s). The STFT uses a periodic Hann window $w[n]=\tfrac12(1-\cos\frac{2\pi n}{N})$, $N=1024$, hop $M=N/2=512$. (a) How many complete frames are computed, and what is the spacing of the spectrogram's time axis? (b) Prove the COLA property $\sum_m w[n-mM]=1$ for this window at $M=N/2$. (c) What constant results at $M=N/4$, and how must overlap-add synthesis account for it?
(a) $N_{\text{frames}}=\bigl\lfloor (L_x-N)/M \bigr\rfloor+1 =\bigl\lfloor (480000-1024)/512\bigr\rfloor+1 = 935+1 = 936$ frames. Time spacing $M/f_s = 512/48000 \approx 10.7$ ms per column.
(b) At hop $M=N/2$, sample $n$ is covered by exactly two shifted windows, at offsets $r$ and $r+N/2$ where $r\equiv n \pmod{N/2}$, $0\le r<N/2$:
where $c=\cos(2\pi r/N)$; the shifted cosine flips sign and the pair sums to exactly 1
for every $n$. (Note this uses the periodic Hann definition with denominator $N$,
exactly the formula in dsp10.m — the "symmetric" Hann with denominator $N-1$
violates COLA slightly.)
(c) At $M=N/4$ each sample is covered by four windows at phases $\theta, \theta+\tfrac\pi2, \theta+\pi, \theta+\tfrac{3\pi}2$. The four cosines cancel in pairs, leaving $\sum_m w[n-mM] = 4\cdot\tfrac12 = 2$. OLA synthesis then returns $y[n]=2x[n]$, so the output must be divided by $C=2$ (in general by $C = \sum_m w[-mM]$) for unit gain.
Problem: Write a MATLAB function myspectrogram(x, N, w, M)
that computes the power spectrogram of a signal $x$ with DFT length $N$, window $w$, and
hop $M$, without using the Signal Processing Toolbox. Use it to analyze an audio recording
with a Hann window of length $N=1024$ and 75% overlap ($M=N/4$): plot the waveform, the
window, and the spectrogram in dB with correctly calibrated time/frequency axes. Report
the frequency resolution. (This is the Lecture-9 homework, dsp10.m.)
Cleaned-up solution code:
clc; clear; close all;
% Step 1: load audio
[x, Fs] = audioread('guitar4.m4a');
t_audio = (0:length(x)-1) / Fs;
figure; plot(t_audio, x, 'r');
xlabel('Time (s)'); ylabel('Amplitude'); title('Audio Signal');
% Step 2: Hann window, N chosen so Fs/N is around 20-40 Hz
N = 2^10; % DFT length (1024)
w = 0.5 * (1 - cos(2*pi*(0:N-1)'/N)); % periodic Hann window
disp(['Frequency resolution (bin spacing): ', num2str(Fs/N), ' Hz']);
figure; plot(w, 'r');
xlabel('Sample Index'); ylabel('Amplitude'); title('Hann Window');
% Step 3-4: spectrogram with hop M = N/4 (75% overlap)
M = N/4;
X = myspectrogram(x, N, w, M);
num_frames = size(X, 2);
t = (0:num_frames-1) * M / Fs; % frame times
f = (0:N/2-1) * Fs / N; % bin frequencies
% Step 5: display in dB
figure;
image(t, f, 10*log10(X(1:N/2, :)));
axis xy;
xlabel('Time (s)'); ylabel('Frequency (Hz)'); title('Spectrogram');
colormap(hot(256)); colorbar;
function X = myspectrogram(x, N, w, M)
L = length(x);
num_frames = floor((L - N)/M) + 1;
X = zeros(N, num_frames);
for m = 1:num_frames
idx = (m-1)*M + (1:N); % frame indices
X(:, m) = abs(fft(x(idx) .* w)).^2; % window, FFT, power
end
end
Key design points:
axis xy puts 0 Hz at the
bottom.Result regenerated below with the identical pipeline (Hann $N=1024$, $M=256$) in NumPy:
myspectrogram pipeline rerun on a synthetic guitar-like signal (three exponentially decaying harmonic "plucks" — G3 196 Hz, B3 246.9 Hz, E4 329.6 Hz — at $t=0,1,2$ s, $f_s=8$ kHz), since the original guitar4.m4a recording is not available here. Each pluck appears as a stack of harmonic lines starting at its onset and fading as the note decays — exactly the structure the homework observes on the real recording.（原始音檔不在手邊，故以合成撥弦訊號重現同一流程；諧波線、起音時刻與衰減皆清晰可見。）