analysis--real-analysis-sequences-series.tex

\section{Sequences and Series}
\footnotetext{Stewart Calculus ch. 11}

\subsection{Definition of sequences and series}

A \textbf{sequence} is an infinite sequence of numbers $a_1, a_2, a_3, ...$.

A \textbf{series} is the sum of a sequence: $s = a_1 + a_2 + a_3 + ... = \sum_{n=1}^\infty a_n$.

A sequence is often defined by giving the closed-form expression for the $\nth$ item, e.g. $a_n = \frac{1}{n}$. This refers to the sequence

\begin{align*}
  \frac{1}{1}, \frac{1}{2}, \frac{1}{3}, ...
\end{align*}

The corresponding series is

\begin{align*}
  \sumn \frac{1}{n} = \frac{1}{1} + \frac{1}{2} + \frac{1}{3} + ...
\end{align*}

A sequence may have a limit: $\lim_{n \rightarrow \infty} a_n$. If it does then the sequence \textbf{converges}. If not, it \textbf{diverges}.

\subsubsection{Typical questions about sequences (11.1)}


\textbf{Determine whether the sequence converges or diverges. If it converges, find the limit.}

- \textbf{Easier} (11.1.24)
\begin{align*}
  a_n = \frac{n^3}{n^3 + 1}
\end{align*}
It's asking whether the sequence converges, so we want to try to compute the limit of the sequence as $n \rightarrow \infty$. If the limit exists then it converges. Rearrange so that all $n$s are on the bottom of fractions:
\begin{align*}
  a_n = \frac{1}{1 + \frac{1}{n^3}}
\end{align*}
So now
\begin{align*}
  \limn a_n = \frac{1}{1 + \limn\frac{1}{n^3}} = \frac{1}{1 + 0} = 1.
\end{align*}

- \textbf{Easier} (11.1.26)
\begin{align*}
  a_n = \frac{n^3}{n + 1}
\end{align*}
With this one, we can't get the $n$s to appear only on the bottom of fractions:
\begin{align*}
  a_n = \frac{n^2}{1 + \frac{1}{n}}
\end{align*}
So
\begin{align*}
  \limn a_n = \frac{\limn n^2}{1 + \limn\frac{1}{n}} = \frac{\limn n^2}{1 + 0} = \limn n^2.
\end{align*}
But $n^2$ increases without bound as $\ninfty$. People write that as $\limn n^2 = \infty$. In any case, the limit does not exist and this sequence diverges.

- \textbf{Harder} (11.1.56)
\begin{align*}
  a_n = \frac{(-3)^n}{n!}
\end{align*}
The negative factor means that the terms of the sequence alternate between being negative and positive. Rewrite it as
\begin{align*}
  a_n = (-1)^n\frac{3}{1}\frac{3}{2}\frac{3}{3}\frac{3}{4}...\frac{3}{n-1}\frac{3}{n}
\end{align*}
Now we use two theorems in the book. The first ("Theorem 6") says that if $\limn |a_n|$ is zero, then so is $\limn a_n$. So that means we can see what happens while ignoring the -1 term. The second theorem is the "Squeeze Theorem": if we can show that the quantity we're studying lies between 0 and something else whose limit is zero, then we've proved that the limit of our quantity is 0. To do that we notice that all the terms from $\frac{3}{4}$ to $\frac{3}{n-1}$ are less than 1. Therefore
\begin{align*}
  0 < |a_n| < \frac{9}{2}\frac{3}{n} = \frac{27}{2n}
\end{align*}
But $\limn \frac{27}{2n} = 0$ so this (together with the Theorem 6 argument about the $(-1)^n$) proves that the sequence converges and that its limiting value is 0.

\subsubsection{Series}

If the series sums to a finite value then it is said to converge; if not, it diverges.

A series has an associated \textbf{sequence} of \textbf{partial sums}: $a_1, a_1 + a_2, a_1 + a_2 + a_3, ...$. I.e. the sequence $a_n = \sum_{i=1}^n a_i$. If the series sums to $s$ (converges) then $s$ is the limit of sequence of partial sums.

Say you only sum the first $n$ values of a series. The \textbf{remainder} $R_n$ is the difference between the true sum (of infinitely many values) and what you got from the first $n$ (the $\nth$ partial sum): $R_n = s - \sum_{i=1}^n$.



In general it is not easy to find the sum of a series, and so a lot of the material in the book focuses on methods for determining whether the series converges or not. However, it is easy to find the sum if the series is a \textbf{geometric series}:

\textbf{Geometric series}
Several questions require recognizing a geometric series. A geometric series is a series where each term differs by a constant multiplier:

\begin{align*}
  s = a + ar^1 + ar^2 + ar^3 + ... = \sumn ar^{n-1}
\end{align*}

Theorem: if $|r| < 1$ then the geometric series converges. Its value is $\frac{a}{1-r}$. For $|r| \geq 1$ it diverges.

\subsubsection{Typical questions about series}

\textbf{Find the values of x for which the series converges. Find the sum of the series for those values of x.}

- 11.2.59
\begin{align*}
  \sum_{n=0}^{\infty} \frac{(x-2)^n}{3^n}
\end{align*}
We have to recognize that this is a geometric series. Rewrite it as
\begin{align*}
  \sum_{n=0}^{\infty} \Big(\frac{x-2}{3}\Big)^n
\end{align*}
and it's clear that it is a GS with $a=1$ and $r=\frac{x-2}{3}$. Therefore it converges if
\begin{align*}
  -1 < \frac{x-2}{3} < 1 \Leftrightarrow -1 < x < 5
\end{align*}
and when x is in that range, its sum is
\begin{align*}
  \frac{a}{1-r} = \frac{1}{1 - \frac{x-2}{3}} = \frac{3}{5-x}.
\end{align*}

\subsubsection{Test for divergence of series}
If the \textbf{sequence} does not converge to zero then the \textbf{series} (the sum of the infinite sequence) is divergent. That's fairly intuitive: if the sequence doesn't converge to zero then the series is going to be summing infinitely many non-zero terms and so will not converge.

So that means that most of the questions about series involve decreasing sequences that converge to zero, for example $a_n = \frac{1}{n}$, or $a_n = \frac{2}{(n-3)^2}$.

What's less intuitive is that even if the sequence \textbf{does} converge to zero, the series still might not converge. The famous example is $\sum_{n=1}^\infty \frac{1}{n} = \frac{1}{1} + \frac{1}{2} + \frac{1}{3} + ...$. Although the sequence converges to zero, its sum (the series) does not converge to a finite value. On the other hand, for $\sum_{n=1}^\infty \frac{1}{n^2} = \frac{1}{1} + \frac{1}{4} + \frac{1}{9} + ...$ the sequence converges to zero and the series does have a finite value. Those facts are not obvious, they require proof: both are infinite sequences of numbers that approach zero, it's just that one aproaches zero more rapidly and has a finite sum whereas the other approaches more slowly and fails to have a finite sum. We can prove it using the \textbf{integral test}.

\subsubsection{Integral test for convergence}
Consider a series like $\sumn \frac{1}{n}$. The continuous function $f(x) = \frac{1}{x}$ passes through the discrete points of the sequence. The integral test says that we can use the area under the graph of the continuous function to assess whether the series converges. That's because we can visualize the series as summing together infinitely many rectangles of width 1, so the sum of the areas of the rectangle is related to the area under the graph.

\textbf{Integral test}: If $\int_1^\infty f(x) dx$ is convergent (has a finite value) then the series is convergent. And if the integral is divergent then the series is divergent.

For example
\begin{align*}
  \int_1^\infty \frac{1}{x^2} dx = \frac{-1}{x} \Big|_1^\infty = 1.
\end{align*}

so that shows that $\sumn \frac{1}{n^2}$ converges (to something, we don't know what). On the other hand

\begin{align*}
  \int_1^\infty \frac{1}{x} dx = \ln x \Big|_1^\infty = \lim_{x \to \infty} \ln x - 0
\end{align*}

which is divergent (not a finite value). So that shows that $\sumn \frac{1}{n}$ does not converge.

\textbf{Remainder theorem for integral test}
Because of the details of the location of the rectangle lines in the approximation, the size of the remainder lies between these two integrals:

\begin{align*}
  \int_{x=n+1}^\infty f(x) dx \leq R_n \leq \int_{x=n}^\infty f(x) dx
\end{align*}


\subsubsection{p-series}

A p-series is a series like $\sumn \frac{1}{n^p}$.

Theorem: a p-series is convergent if $p>1$ and divergent if $p \leq 1$.

\subsection{Comparison tests}

\subsubsection{Comparison to known-convergent series}
Consider a series for which all terms are \textbf{positive}. If we can show that the terms of our series are always less than the corresponding terms of a known-convergent series, then our series is convergent. SImilarly if our terms are larger than those of a known-divergent series, then our series is divergent.

For example, we know that $\sumn\frac{1}{2^n}$ converges to zero (p-series with $p > 1$) but what about $\sumn\frac{1}{2^n + 1}$? It seems obvious that it converges to zero, but how do you prove it? The answer is that its terms are always smaller than the terms of $\sumn\frac{1}{2^n}$, therefore it also converges to zero.


\subsubsection{Limit comparison test}
This involves looking at the limit of the \textbf{ratio} of the terms from two different series. Suppose you have a series $\sumn b_n$ which you know either converges or diverges, and you have a series $\sumn a_n$ which you don't know the behavior of. If you can show that their ratio has a finite limit:

\begin{align*}
  \limn \frac{a_n}{b_n} = c > 0
\end{align*}
then either \textbf{both converge} or \textbf{both diverge}. Seeing as you know how $b$ behaves, that tells you whether $a$ converges or not.


\subsection{Alternating series}
The two theorems about alternating series are fairly obvious if we draw a picture of an alternating series.

\subsection{Alternating series test}
The previous tests are all for series with positive terms only. Alternating series are often the result of a $(-1)^n$ factor, for example $\sumn (-1)^n \frac{1}{n}$. The \textbf{alternating series test} says: look at the \textbf{absolute values}; if they are decreasing (i.e. $|a_{n+1}| < |a_n|$), and if their limit is zero, then the series converges.


\subsubsection{Alternating series remainder theorem}
The absolute size of the error is less than or equal to the size of the first neglected term. (Only for alternating series!)


\subsection{Absolute and conditional convergence}

Sometimes a series with positive and negative terms might converge, but the absolute value version of the same series does \textbf{not} converge. This situation is called \textbf{conditional convergence}. An example is $\sumn (-1)^n \frac{1}{n}$. This converges (by alternating series test: the successive terms get smaller) but the absolute version doesn't converge (as discussed above).

If the absolute value version of an alternating series \textbf{does} converge, then this is called \textbf{absolute convergence}.

Theorem: if an alternating series is absolutely convergent, then the alternating version is convergent also.

In other words, there are two different categories of convergent alternating series: conditional (only the alternating version converges), and absolute (both versions converge).

\subsection{Ratio and root tests}

These can be used for alternating and non-alternating series.

\subsubsection{Ratio test}
Write down the ratio of successive terms in the series $\frac{a_{n+1}}{a_n}$, take the absolute value and take the limit of that. So $\limn \Big|\frac{a_{n+1}}{a_n}\Big|$. There are 3 possible outcomes:

\begin{align*}
  \begin{cases}
    0 < \text{limit} < 1&Convergent\\
    \text{limit} = 1&Inconclusive\\
    \text{limit} > 1 ~\text{or}~ \text{limit} = \infty&Divergent\\
  \end{cases}
\end{align*}

If the original series was alternating, then these conclusions of "convergent" and "divergent" apply to the alternating series too, as well as the series of absolute values.


\subsubsection{Root test}
This is like the ratio test, but you use it if there are powers of $n$.

Evaluate the limit of the absolute value of the $\nth$ root of the $\nth$: $|\sqrt[n]{a_n}|$. The cases are the same as for the ratio test:


\begin{align*}
  \begin{cases}
    0 < \text{root} < 1&Convergent\\
    \text{root} = 1&Inconclusive\\
    \text{root} > 1 ~\text{or}~ \text{root} = \infty&Divergent\\
  \end{cases}
\end{align*}



\subsection{Strategy}

See section 11.7 for overall strategy for testing series for convergence/divergence.


\subsection{Power series}

\subsubsection{Geometric series}

One class of tricks involves connecting the function to a geometric series. For example,

\subsubsection{Taylor and Maclaurin series}

Memorize the general formula for a Taylor series around a point $a$. (Maclaurin series is the same but with $a=0$).

A common question is: over what interval of $x$-values does the series converge? This can be answered using the Ratio Test for convergence.

\subsubsection{Maclaurin series derivation}

Suppose that any function of a real number $f(x)$ can be represented by a "power series" with certain coefficients $c_i$

\begin{align*}
  f(x) = c_0 + c_1x^1 + c_2x^2 + c_3x^3 + c_4x^4 + ...
\end{align*}

If that is so, there are two questions:

(1) what are the coefficients for any given function? This can be established
by differentiating and evaluating the result at $x=0$.

(2) Over what interval of $x$-values does the series converge? This can be
answered using the Ratio Test for convergence.

To answer (1), firstly, without differentiating at all, we see that $c_0 = f(0)$. Then differentiate once:

\begin{align*}
  f'(x) = c_1 + 2c_2x^1 + 3c_3x^2 + 4c_4x^3 + ...
\end{align*}

and we see that evaluating the first derivative at $x=0$ gives the coefficient
$c_1 = f'(0)$. Differentiating again yields $c_2$:

\begin{align*}
  f''(x) &= (2\cdot 1)c_2 + (3\cdot 2)c_3x^1 + (4\cdot 3)c_4x^2 + ... \\
  \frac{f''(0)}{2} &= c_2
\end{align*}

Continuing like that it becomes clear that each coefficient is equal to an $\nth$ derivative evaluated at zero, divided by a factorial term which results from the repeated differentiation:

\begin{align*}
  f'''(x) &= (3\cdot2)c_3 + (4\cdot3\cdot2)c_4x^1 + ... \\
  \frac{f'''(0)}{3!} &= c_2
\end{align*}

and in general $c_n = \frac{f^{(n)}(0)}{n!}$.

For example, take $f(x) = e^x$. The derivatives are all the same of course: $f^{(n)}(x) = e^x$. And $e^0 = 1$, so the Maclaurin expansion of $e^{x}$ is
\begin{align*}
  e^x = e^{0} + \frac{xe^0}{1!} + \frac{x^2e^0}{2!} + \cdots = \sum_{n=0}^\infty \frac{x^n}{n!}
\end{align*}
The Maclaurin expansion of $e^{-x}$ is
\begin{align*}
  e^{-x} = e^{0} - \frac{xe^0}{1!} + \frac{x^2e^0}{2!} - \cdots = \sum_{n=0}^\infty (-1)^{n}\frac{x^n}{n!}.
\end{align*}

\begin{align*}
  \lim_{n \to \infty} \(1 + \frac{x}{n}\)^n
  &= \lim_{n \to \infty} \sum_{k=0}^n {n \choose k} \(\frac{x}{n}\)^k \\
  &= \lim_{n \to \infty} \sum_{k=0}^n \frac{n!}{(n-k)!k!} \(\frac{x}{n}\)^k \\
\end{align*}
... \red{TODO} prove this is $e^x$.




\section{Exercises}

- \textbf{11.1: 3, 25, 37}
- \textbf{11.2: (14, 29, 42)}
- One trick to find sums: use partial fractions and then write first few terms of series out and identify a telescoping sum (q. 14)

- \textbf{11.3: (7, 16, 25, 26) Determine whether convergent or divergent (Integral test)}
- Use integral test (if $\int_1^\infty f(x) dx$ is finite then converges, otherwise diverges)
- Although it seems less work to use limit comparison test for some

- \textbf{11.4: (18, 25, 26) Determine whether convergent or divergent}
- Questions concern series with positive terms
- Use Comparison Test or Limit Comparison Test
- $\frac{1}{n}$ diverges. So if you can show that terms are larger than this, or that limit of ratio with this exists, then it must also be divergent.
- $\frac{1}{n^2}$ converges. If you can show that terms are smaller than this, or that limit of ratio with this exists, then it converges.

- \textbf{11.5 (5, 8, 11) Determine whether convergent or divergent}
- Questions typically concern alternating series
- Alternating series test: show that successive values are decreasing \textbf{and} that limit of sequence is 0.
- If it's not obviously decreasing, convert $a_n$ to continuous function $f(x)$ and show that derivative is negative or becomes negative for large values of $x$.

- \textbf{11.6 (5, 6, 29) Absolutely convergent | Conditionally convergent | Divergent}
- Questions typically concern alternating series
- Use Ratio Test on absolute values to check if convergent or divergent. If it is convergent on the Ratio Test then it is "absolutely convergent".
- If Ratio Test is inconclusive (limit $= 1$) then the Alternating Series Test might show that it is Conditionally Convergent.

- \textbf{11.8 (10, 12) Find radius / interval of convergence}
- Typically, use the ratio test (converges for values of $x$ that make $\limn |\frac{a_{n+1}}{a_n}|$ less than 1)
- If they ask for interval, then you need to test the endpoints for convergence separately.

- \textbf{11.9 (3, 6) Find a power series representation for a given function, and radius/interval of convergence.}
- Match up the function to the formula for the sum of a geometric series ($a/(1-r)$) and thus construct a geometric series that it's equal to.
- Use ratio test to determine interval of convergence
- A variant is: you have to differentiate the function before it looks like $a/(1-r)$. Then find the power series, and integrate the power series to get something equal to the original function.

- \textbf{11.10 (8, 9, 17) Find a Taylor / Maclaurin series for a given function, and radius/interval of convergence}
- Memorize definition of general Taylor series (Maclaurin is $a=0$)
- Compute derivatives and evaluate them at zero
- Use those to write out first few terms of Taylor series
- If there's a pattern, represent the sum using sigma notation
- Use the ratio test to assess radius of convergence