The inventors of calculus in the seventeenth century did not have rigorous definitions of limits and continuity; these were achieved only in the 1870s. Rigor is ulti- mately necessary in mathematics, but it does not always come first, as Archimedes acknowledged in a manuscript discovered in 1906.
In it Archimedes reveals that his deepest results were found us- ing dubious arguments, and only later proved rigorously, because
"it is of course easier to supply the proof when we have previously acquired some knowledge of the questions by the method, than it is to find it without any previ- ous knowledge." (We found this story in John Stillwell's Mathe- matics and Its History, Springer Verlag, 1997.)
FIGURE 1.5.1.
An open set includes none of the fence; however close a point in the open set is to the fence, you can always surround it with a ball of other points in the open set.
FIGURE 1.5.2.
A closed set includes its fence.
84 Chapter 1. Vectors, matrices, and derivatives
hardest and deepest of all of mathematics; mathematicians struggled for two hundred years to come up with correct definitions. More students have foundered on these definitions than on anything else in calculus: the combination of Greek letters, precise order of quantifiers, and inequalities is a hefty obstacle. Fortunately, these notions do not become more difficult in several variables than they are in one variable. Working through a few examples will help you understand what the definitions mean, but a proper appreciation can probably only come from use; we hope you have already started on this path in one-variable calculus.
Open and closed sets
In mathematics we often need to speak of an open set U; whenever we want to approach points of a set U from every side, U must be open.
Think of a set or subset as your property, surrounded by a fence. The set is open if the entire fence belongs to your neighbor. As long as you stay on your property, you can get closer and closer to the fence, but you can never reach it; no matter how close you are to your neighbor's property, there is an epsilon-thin buffer zone of your property between you and it.
The set is closed if you own the fence. Now, if you sit on your fence, there is nothing between you and your neighbor's property. If you move even an epsilon further, you will be trespassing.
What if some of the fence belongs to you and some belongs to your neighbor? Then the set is neither open nor closed.
To state this in proper mathematical language, we first need to define an open ball. Imagine a balloon of radius r, centered around a point x. The open ball of radius r around x consists of all points y inside the balloon, but not the skin of the balloon itself. We use a subscript to indicate the radius of a ball B; the argument gives the center of the ball: B2(a) is a ball of radius 2 centered at the point a.
Definition 1.5.1 (Open ball). For any x E IR.n and any r > 0, the open ball of radius r around x is the subset
Br(x) ~ {y E IR.n such that Ix -yl < r}. 1.5.l Note that Ix - YI must be less than r for the ball to be open; it cannot be = r. In dimension 1, a ball is an interval.
Definition 1.5.2 (Open set of IR.n). A subset UC IR.n is open if for every point x E U, there exists r > 0 such that the open ball Br ( x) is contained in U.
An open set is shown in Figure 1.5.1, a closed set in Figure 1.5.2.
Example 1.5.3 (Open sets).
1. If a < b, then the interval (a, b) = { x E JR. I a < x < b } is open.
Indeed, if x E (a, b), set r = min { x - a, b - x}. Both these numbers
Parentheses denote an open in- terval and brackets denote a closed one: (a, b) is open, [a, b] is closed.
Sometimes backwards brackets are used to denote an open inter- val: ]a,b[= (a,b).
Open and closed subsets of nr
are special; you shouldn't expect a subset to be either. For in- stance, the set of rationals Q C JR is neither open nor closed: every neighborhood of a rational num- ber contains irrational numbers, and every neighborhood of an ir- rational number contains rational numbers.
D FIGURE I~ 1.5.3.
The natural domain of
is neither open nor closed. It in- cludes the y-axis (with the origin removed) but not the x-axis.
are strictly positive, since a < x < b, and so is their minimum. Then the ball { y I IY-xi< r} is a subset of (a,b).
2. The rectangle
(a, b) x ( c, d) = { ( ~) E IR2 I a < x < b , c < y < d } is open. 1.5.2 3. The infinite intervals (a, oo), ( -oo, b) are also open, but the intervals (a, b] = { x E JR I a < x :::; b} and [a, b] = { x E JR I a :::; x :::; b} 1.5.3
are not. 6.
A door that is not open is closed, but a set that is not open is not necessarily closed. An open set owns none of its fence. A closed set owns all of its fence:
Definition 1.5.4 (Closed set of !Rn). A subset Cc !Rn is closed if its complement !Rn - C is open.
We discussed the natural domain (the "domain of definition") of a for- mula in Section 0.4. We will often be interested in whether this domain is open or closed or neither.
Example 1.5.5 (Natural domain: open or closed?). Is the natural domain of the formula
f ( ~) = ~ open or closed? 1.5.4
If the argument of the square root is nonnegative, the square root can be evaluated, so the first and the third quadrants are in the natural domain.
The x-axis is not (since y = 0 there), but the y-axis with the origin removed is in the natural domain, since x/y is zero there. So the natural domain is the region drawn in Figure 1.5.3; it is neither open nor closed. 6.
Remark 1.5.6 (Why we specify open set). Even very good students often don't see the point of specifying that a set is open. But it is absolutely essential, for example in computing derivatives. If a function f is defined on a set that is not open, and thus contains at least one point x that is part of the fence, then talking of the derivative of f at x is meaningless.
To compute f' ( x) we need to compute
J'(x) = lim -h1 (f(x + h) - f(x)),
h-+O 1.5.5
but f ( x + h) won't necessarily exist even for h arbitrarily small, since x + h may be outside the fence and thus not in the domain of f. This situation gets much worse in !Rn. 10 6.
10For simple closed sets, fairly obvious ad hoc definitions of derivatives exist.
For arbitrary closed sets, one can also make sense of the notion of derivatives, but these results, due to the great American mathematician Hassler Whitney, are extremely difficult, well beyond the scope of this book.
A set is open if it contains a neighborhood of every one of its points.
An open subset U c IRn is
"thick" or "chunky". No matter where you are in U, there is al- ways a little room: the ball Br(x) guarantees that you can stretch out at least r > 0 in any di- rection possible in your particular
!Rn, without leaving U. Thus an open subset U C !Rn is necessar- ily n-dimensional. In contrast, a plane in IR3 cannot be open; a flat- worm living in the plane cannot lift its head out of the plane. A line in a plane, or in space, cannot be open; a "lineworm" living in a line that is a subset of a plane or of space can neither wiggle from side to side nor lift its head.
A closed n-dimensional subset of IRn is "thick" in its interior (if the interior isn't empty) but not on its fence. If an n-dimensional subset of !Rn is neither closed nor open, then it is "thick" everywhere except on the part of fence that it owns.
Equation 1.5.8: Remember that U denotes "union": A U B is the set of elements of either A or B or both. The notation of set theory is discussed in Section 0.3.
86 Chapter 1. Vectors, matrices, and derivatives
Neighborhood, closure, interior, and boundary
Before discussing the crucial topics of this section - limits and continuity - we need to introduce some more vocabulary.
We will use the word neighborhood often; it is handy when we want to describe a region around a point without requiring it to be open; a neighborhood contains an open ball but need not be open itself.
Definition 1.5.7 {Neighborhood). A neighborhood of a point x E !Rn is a subset X C !Rn such that there exists E > 0 with Be(x) C X.
Most often, we deal with sets that are neither open nor closed. But every set is contained in a smallest closed set, called its closure, and every set contains a biggest open set, called its interior. (Exercise 1.5.4 asks you to show that these characterizations of closure and interior are equivalent to the following definitions.)
Definition 1.5.8 (Closure). If A is a subset of !Rn, the closure of A, denoted A, is the set of x E !Rn such that for all r > 0,
1.5.6 Definition 1.5.9 (Interior). If A is a subset of !Rn, the interior of A,
0
denoted A, is the set of x E !Rn such that there exists r > 0 such that
Br(x) CA. 1.5.7
The closure of a closed set is itself; the interior of an open set is itself.
We spoke of the "fence" of a set when we defined closed and open sets informally. The technical term is boundary. A closed set contains its bound- ary; an open set contains none of its boundary. We used the word "fence"
because we think it is easier to think of owning a fence than owning a boundary. But "boundary" is generally more appropriate. The boundary of the rational numbers is all of JR, which would be difficult to imagine as a picket fence.
Definition 1.5.10 (Boundary of subset). The boundary of a subset A c !Rn, denoted a A, consists of those points x E !Rn such that every neighborhood of x intersects both A and the complement of A.
The closure of A is thus A plus its boundary; its interior is A minus its boundary:
0
A=AuaA and A= A-aA. 1.5.8
The boundary is the closure minus the interior:
- 0
aA =A-A. 1.5.9
Exercise 1.18 asks you to show that the closure of U is the subset of nr made up of all limits of sequences in U that converge in Rn.
Exercise 1.5.6 asks you to show that aA is the intersection of the closure of A and the closure of the complement of A.
FIGURE 1.5.4. The shaded region is U in Ex- ample 1.5.11, part 5. You can ap- proach the origin from this region, but only in rather special ways.
FIGURE 1.5.5.
Karl Weierstrass (1815-1897) At his father's insistence, Weier- strass studied law, finance, and economics, but he refused to take his final exams. Instead he be- came a teacher, teaching mathe- matics, history, physical education and handwriting. In 1854 an arti- cle on Abelian functions brought him to the notice of the mathe- matical world.
Exercise 1.5.4 asks you to justify equations 1.5.8 and 1.5.9.
Examples 1.5.11 (Closure, interior, and boundary).
1. The sets (0, 1), [O, 1], [O, 1), and (0, 1] all have the same closure, interior, and boundary: the closure is [O, 1], the interior is (0, 1), and the boundary consists of the two points 0 and l.
2. The sets
{ ( ~) E JR2 I x2 + y2 < 1 } and { ( ~) E JR2 I x2 + y2 :::; 1 } both have the same closure, interior, and boundary: the closure is the disc of equation x2 + y2 :::; 1, the interior is the disc of equation x2 + y2 < 1, and the boundary is the circle of equation x2 + y2 = l.
3. The closure of the rational numbers Q C JR is all of JR: the intersec- tion of every neighborhood of every real number with Q is not empty.
The interior is empty, and the boundary is JR: every neighborhood of every real number contains both rationals and irrationals.
4. The closure of the open unit disc with the origin removed:
U = { ( ~) E JR2 I 0 < x2 + y2 < 1 } 1.5.10 is the closed unit disc; the interior is itself, since it is open; and its boundary consists of the unit circle and the origin.
5. The closure of the region U between two parabolas touching at the origin, shown in Figure 1.5.4:
1.5.11 is the region given by
1.5.12
0
in particular, it contains the origin. The set is open, so U = U. The boundary is the union of the two parabolas. 6
Convergence and limits
Limits of sequences and limits of functions are the fundamental construct of calculus, as you will already have seen in first year calculus: both derivatives and integrals are defined as limits. Nothing can be done in calculus without limits.
The notion of limit is complicated; historically, coming up with a correct definition took 200 years. It wasn't until Karl Weierstrass (1815-1897) wrote an incontestably correct definition that calculus became a rigorous subject.11 A major stumbling block to writing an unambiguous definition
11 Many great mathematicians wrote correct definitions of limits before Weier- strass: Newton, Euler, Cauchy, among others. However, Weierstrass was the first to show that his definition, which is the modern definition, provides an adequate foundation for analysis.
Definition 1.5.12: Recall from Section 0.2 that 3 means "there exists" ; \;/ means "for all" . In words, the sequence i >--+ ai con- verges to a if for all € > 0 there exists M such that when m > M, then lam - al < €.
Saying that i >--+ ai converges to 0 is the same as saying that
lim;~oo jail= O; we can show that a sequence of vectors converges to 0 by showing that their lengths converge to 0. Note that this is not true if the sequence converges to a i= 0.
Infinite decimals are limits of convergent sequences. If
ao = 3, ai = 3.1,
a2 = 3.14,
an = 7r to n decimal places, how large must M be so that if n ~ M, then Ian - trl < 10-3? The answer is M = 3:
7r - 3.141 = .0005926 ....
The same argument holds for any real number.
88 Chapter 1. Vectors, matrices, and derivatives
was understanding in what order quantifiers should be taken. (You may wish to review the discussion of quantifiers in Section 0.2.)
We will define two notions of limits: limits of sequences and limits of functions. Limits of sequences are simpler; for limits of functions one needs to think carefully about domains.
Unless we state explicitly that a sequence is finite, it is infinite.
Definition 1.5.12 (Convergent sequence; limit of sequence). A sequence i ~ 3i of points in IRn converges to a E IRn if
\/€ > 0, 3M I m > M = } la.n - al < €. 1.5.13 We then call a the limit of the sequence.
Exactly the same definition applies to a sequence of vectors: just replace a in Definition 1.5.12 by a, and substitute the word "vector" for "point".
Convergence in !Rn is just n separate convergences in JR:
Proposition 1.5.13 (Convergence in terms of coordinates). A sequence m ~ a.n with a.n E JRn converges to a if and only if each coordinate converges; i.e., if for all j with 1 :::; j :::; n the jth coordinate (ai)j of 3i converges to ai, the jth coordinate of the limit a.
The proof is a good setting for understanding how the (t:, M)-game is played (where M is the M of Definition 1.5.12). You should imagine that your opponent gives you an epsilon and challenges you to find an M that works: an M such that when m > M, then l(am)j - ail < t:. You get Brownie points for style for finding a small M, but it is not necessary in order to win the game.
Proof. Let us first see the easy direction: that m ~ a.n converges to a
~:.[::~~.::: ::~ ~ul:~ .::[::• ::.::::t(e (~::J, c)on::::::,::y
(am)n
you have a teammate who knows how to play the game for the sequence m ~ a.n, and you hand her the epsilon you just got. She promptly hands you back an M with the guarantee that when m > M, then lam - al < t:
(since the sequence m ~ am is convergent). The length of the vector
a.n - a is
lam - al= V ((amh - ai)2 + ã ã ã + ((am)n - an)2, so you give that M to your challenger, with the argument that
He promptly concedes defeat.
1.5.14
1.5.15
This is typical of all proofs involving convergence and limits:
you are given an € and challenged to come up with a o (or M or whatever) such that a certain quantity is less than €.
Your "challenger" can give you
any € > 0 he likes; statements
concerning limits and continuity are of the form "for all epsilon, there exists . . . " .
Now we have to show that convergence of the coordinate sequences im- plies convergence of the sequence m f-+ am. Again the challenger hands you an E > 0. This time you have n teammates; each knows how the play the game for a single convergent coordinate sequence m f-+ (am )j. After a bit of thought and scribbling on a piece of paper, you pass along i:/ .Jii,
to each of them. They dutifully return to you cards containing numbers M1 , ... , Mn, with the guarantee that
l(am)j - ajl < Vn E when m > Mj. 1.5.16 You sort through the cards and choose the one with the largest number,
M = max{M1, ... ,Mn}, 1.5.17
which you pass on to the challenger with the following message:
if m > M, then m > Mj for each j = 1 ã ã ã = n, so l(am)j - ajl < i:/ .Jii,,
so
1.5.18 0
The scribbling you did was to figure out that handing i:/ .Jii, to your teammates would work. What if you can't figure out how to "slice up" E so that the final answer will be precisely i:? Just work directly with E and see where it takes you. If you use E instead of i::/.Jii, in inequalities 1.5.16 and 1.5.18, you will end up with
lam - al < €Vn. 1.5.19
You then see that to get the exact answer, you should have chosen i:/.Jii,.
In fact, the answer in inequality 1.5.19 is good enough and you don't need to go back and fiddle. "Less than epsilon" for any E > 0 and "less than some quantity that goes to 0 when epsilon goes to O" achieve the same goal:
showing that you can make some quantity arbitrarily small. The following theorem states this precisely; you are asked to prove it in Exercise 1.5.11.
The first statement is the one mathematicians use most often.
Proposition 1.5.14 (Elegance is not required). Let u be a function of E > 0 such that u(i:) - 0 as E - 0. Then a sequence if-+ a,; converges to a if either of the two equivalent statements are true:
1. For all E > 0 there exists M such that when m > M, then lam - al< u(i:).
2. For all E > 0 there exists M such that when m > M, then lam-al< i:.
The following result is of great importance. It says that the notion of limit is well defined: if the limit is something, then it isn't something else.
Theorem 1.5.16, part 4: "If Sm
is bounded" means "if there exists
R < oo such that lsml ~ R for all
m". To see that this requirement is necessary, consider
Cm = 1/m and Sm = ( ~) .
Then m ,... Cm converges to 0, but Jim (cmsm) =/= 0.
m-oo
This does not contradict part 4, because Sm is not bounded.
Proposition 1.5.17 says that there is an intimate relationship between limits of sequences and closed sets: closed sets are "closed under limits".
The second part of Proposition 1.5.17 is one possible definition of a closed set.
90 Chapter 1. Vectors, matrices, and derivatives
Proposition 1.5.15 (Limit of sequence is unique). If the sequence i 1-+ 3i of points in ~n converges to a and to b, then a = b.
Proof. This could be reduced to the one-dimensional case, but we will use it as an opportunity to play the ( E, M)-game in more sober fashion. Suppose a =f. b, and set Eo =(la - bl)/4; our assumption a =f. b implies that Eo > 0.
By the definition of the limit, there exists Mi such that lam - al < Eo when m > Mi, and M2 such that lam - bl < Eo when m > M2. Set M =max{ Mi, M2}ã If m > M, then by the triangle inequality,
la- bl= l(a - am)+ (am - b)I ~la ...___.., - aml +lam - bl< "'-...--' 2Eo = ls-2 bl .
<eo <eo
This is a contradiction, so a = b. D
Theorem 1.5.16 (The arithmetic of limits of sequences). Let i 1-+ 3i and i 1-+ bi be two sequences of points in ~n, and let i 1-+ ci be a sequence of numbers. Then
1. If i 1-+ 3i and i 1-+ bi both converge, then so does i 1-+ 3i +bi, and Hm (3i +bi) = _lim 3i + Hm bi. 1.5.20
i-+oo t-+oo i-+oo
2. If i 1-+ 3i and i 1-+ ci both converge, then so does i 1-+ Ci3i, and nm Ci3i = ( _lim Ci) ( _lim 3i) . 1.5.21
t-+00 t--+00 t-+00
3. If the sequences i 1-+ 3i and i 1-+ bi converge, then so do the sequences of vectors i 1-+ ~ and i 1-+ bi, and the limit of the dot products is the dot product of the limits:
_lim (~ ã bi)= (nm~) ã (nm bi). 1.5.22
i-+oo i-+oo t-+oo
4. If i 1-+ 3i is bounded and i 1-+ Ci converges to 0, then lim Ci3i = 0.
i-+oo 1.5.23
Proof. Proposition 1.5.13 reduces Theorem 1.5.16 to the one-dimensional case; the details are left as Exercise 1.5.17. D
Proposition 1.5.17 (Sequence in closed set).
1. Let i 1-+ Xi be a sequence in a closed set C C ~n converging to xo E ~n. Then xo EC.
2. Conversely, if every convergent sequence in a set C C ~n con- verges to a point in C, then C is closed.
Intuitively, this is not hard to see: a convergent sequence in a closed set can't approach a point outside the set; in the other direction, if every