Estimating through Monte Carlo Methods in n-dimensions

Figure 4: Construction of a vertex of the superscribed hexagon.Figure 5 shows the circle with the inscribed and superscribed hexagons.. Figure 8: Construction of a new vertex of the insc

Estimating  through Monte Carlo Methods in n-dimensions

Brian Reece Iowa State UniversityMSM Creative ComponentJune 30, 2006

Heather Thompson, Co-Major Professor

Irvin R Hentzel, Co-Major Professor

Alejandro Andreotti, Committee Member

The number  has intrigued mathematicians for more than 4000 years In the pursuit of , mathematicians have used geometry, trigonometry, algebra, calculus and probability This interaction between different branches of mathematics is one of things that makes  interesting to study This paper begins by looking at different ways of calculating that have been used throughout history Next, it looks at the Monte Carlo method and its applications Next, a method of estimating  with Monte Carlo methods

in n-dimensional space is presented The paper ends with a discussion of how the topics can be incorporated into the classroom

The number  has not always had a name It wasn’t until 1706, when William Jones wrote3.14159 , that the world had a symbol for the ratio of a circle’s

circumference to its diameter Euler adopted the symbol in 1737 and it became the standard symbol

The ratio of a circle’s circumference to its diameter has long been understood as a constant

Then he made the molten sea; it was round, ten cubits from brim to brim,

and five cubits high, and a line of thirty cubits measure its circumference

(I Kings 7:23)

This passage from the Bible places that constant at30 3

10  This estimate was mostlikely achieved through measurement, so it is quite reasonable given the likely precision

of measurement An Egyptian document called the Rhind papyrus, which dates to 1650

, or 3.1605 It was written by a scribe named Ahmes, who

copied it from a document that is 200 years older Some even believe that the Rhind papyrus can trace its origin back to 3400 B.C

Sets of numbers

To understand what makes  special, one should look at where it fits into the realm of all numbers The number  is a transcendental number The transcendental numbers are so called because they transcend the algebraic numbers

Algebraic numbers are roots of a polynomial equation of the form

a X a X  a X   a X a   , where a are integers i

For example, 2 , an irrational number, is a solution to the quadratic equation

x   Algebraic numbers also include complex numbers Using the simplest

example, i is a solution of the polynomial equation x2  1 0

Transcendental numbers are not the root of any integer polynomial equation In

1844, Joseph Liouville proved that transcendental numbers exist by finding examples Here is an example of one such number, called Liouville’s constant in honor of its

This number contains 0’s in every place except for the n! decimal places So the 1st, 2nd,

6th, 24th, … decimal places contain 1’s The next 1 will occur in the 5! = 120th decimal

place In 1850, Liouville proved that this number cannot be a solution of a polynomial equation with integer coefficients and thus is a transcendental number

The number e was proven to be transcendental in 1873 by Charles Hermite

Ferdinand Lindemann used Hermite’s results to prove that  was transcendental in 1882.This proof by Lindemann also had a powerful byproduct Since is transcendental, it follows that  cannot be the solution to a polynomial equation with integer coefficients; therefore, one of the earliest math mysteries was proven an impossibility

Since antiquity, mathematicians had tried to construct a square of area equal to thearea of a given circle using only a straightedge and compass Lindemann’s discovery that

 is not the root of any polynomial equation proves this exercise impossible In order to

do this, one must construct  , which is only a possibility if  is an algebraic number With Lindemann proving that  is a transcendental number, he showed that the problem

of squaring a circle was unsolvable in Euclidean geometry

Archimedes’ Method of Trapping 

Archimedes was the first to develop a formal approximation of He constructed two series of polygons, one that inscribed a given circle and one that superscribed the given circle Using these polygons he created an interval for

Archimedes first showed that the area of a regular polygon is

1

2apothem perimeter An n-gon would have exactly n congruent triangles drawn from the circumcenter of the polygon to each vertex Figure 1 shows the first three regular n- gons divided into n congruent triangles.

Figure 1: Regular n-gons divided into n congruent triangles.

The area of a triangle is, of course1

2bh When this formula is applied to one of

the congruent triangles, the area of the triangle is 1

2

T

A  sa , where s is a side length and

a is the apothem The apothem is the perpendicular distance from the center to one side

of the polygon The area of the entire n-gon can be found then by multiplying this area by

n , resulting in

12

n gon

A  n sa

The perimeter of a regular n-gon is just ns , so substituting results in

12

n gon

A  ap

Figure 2 helps illustrate that as n   , aR and p , and thereforeC

12

circle

A  RC.This result was Archimedes’ formula for the area of a circle

Figure 2: Regular triangle, square and 30-gon with apothem and radius of the

circumscribed circle marked

From Euclid, Archimedes knew that the ratio of a circle’s area to the square of its diameter was a constant Let k be that constant Then

2

Area k

 4C d,

which results in C 4k

d  (Linn and Neal, March 2006)

This constant 4k is of course now known as

Consider the unit circle The diameter would be 2 and 1

and superscribe a regular polygon of 3 2 sides, with semiperimeter n b By constructing n

these regular polygons, one obtains an increasing sequence a a a1, , , 2 3  and a decreasingsequence b b b1, , , 2 3  such that as n ,a n  and b n  This traps  between two numbers,a n    b n

Starting withn , one has a regular polygon with 6 sides, or a hexagon 1

Choosing a point A on the circle and copying the radius, one can find two more points of the inscribed hexagon by drawing an arc centered at A Figure 3 shows the results of this construction, points B and D Repeating this process with each resulting point as a center

for the arc results in the inscribed hexagon

R

D

B

Figure 3: Construction of vertices of inscribed hexagon.

To get the superscribed hexagon, a line is constructed perpendicular to the radius

at each of the vertices of the inscribed hexagon Where the perpendicular lines intersect are the vertices of the superscribed hexagon Figure 4 illustrates the construction of one

such vertex Point H is a vertex of the superscribed hexagon.

Figure 4: Construction of a vertex of the superscribed hexagon.

Figure 5 shows the circle with the inscribed and superscribed hexagons

Figure 5: Archimedes’s first attempt at trapping

A side length in the inscribed hexagon would be 1 So the perimeter is 6, anda1  3Figure 6 shows the length of one half-side of the inscribed hexagon expressed in terms of

a trigonometric ratio Figure 7 shows a similar result for the superscribed hexagon

s

2=cos60° 60°

R=1

s

2=cot60° 

R=1

Figure 7: Superscribed regular hexagon with radius and half-side length represented.

If n is increased to 2, a dodecagon must be constructed First, one has to draw a ray from the center, O, to a vertex of the superscribed hexagon The point of intersection

between this ray and the circle is a point on the inscribed dodecagon Drawing 6 of these rays results in 6 new points, or 12 in total These 12 points are the vertices of the

inscribed dodecagon Figure 8 illustrates the construction of one of these vertices

Figure 8: Construction of a new vertex of the inscribed regular dodecagon.

Figure 9 illustrates the construction of a vertex of the superscribed dodecagon

First the midpoint of HC and the midpoint ofGC are constructed Then two rays are drawn from the center of the circle, O, through points B and D The point of intersection

of OBuuur and O e is point A and the intersection ofODuuurand O e is point E Next a line perpendicular to OAuuur is constructed through point A; the same is done for OEuuur through

point E The intersection of these two lines is point I, a vertex on the superscribed

F

H

G

Figure 9: Construction of a vertex on the superscribed regular dodecagon.

This process can be repeated from each 3 2 -gon to gain vertices of the n 1

3 2 n gons

-In general terms a and n b can be expressed as n

What makes Archimedes’ work incredible is that he did not have the use of

trigonometry or algebra to develop his interval for These had not been developed before his lifetime Beginning with a hexagon and increasing the number of sides by a

factor of two, Archimedes used a 96-gon, when n=5, to obtain this interval:223 22

71   7 For many years Archimedes’s method remained the only formal method for calculating The next section looks at mathematicians’ attempts to calculate  using infinite series

Gregory’s Infinite Series

James Gregory (1638-1675) and Gottfried Leibniz (1646-1716) both

independently developed an infinite series to calculate  They used the Taylor series

expansion of arctangent to find an infinite series that equals

4

 The Taylor series

polynomial for arctangent is

Proof That the Series Converges

The alternating series test, or the Leibniz criterion, states that for an alternating

series of the form

1

( 1)n n n

Table 1 shows the partial sums for the first n terms of the Gregory/Leibniz series

More Gregory/Leibniz, John Machin and Machin-Like Formulae

In 1706 John Machin improved upon the Gregory/Leibniz series by finding a series with an increased rate of convergence The formula that Machin developed was

is 17 digits However, with rounding error the optimal number of decimal places is actually 16.

The current record for the most digits of , 1,241,100,000,000, was obtained in

2002 by Yasumasa Kanada of Tokyo University Kanada used a 64-node Hitachi

supercomputer with 1 terabyte of main memory to evaluate a Machin-like formula

following section discussion proceeds as to how and why simulations can be used to estimate

Monte Carlo Method

A Monte Carlo method can be described as a method that solves a problem by generating suitable random numbers and noting that proportion of the numbers obeying some property Named after the city in Monaco known for its casinos, the method was dubbed Monte Carlo method by Nick Metropolis Credit for the invention of the method generally goes to Stanislaw Ulam, but there is evidence of its use before the 1940s when

it was formalized

Ulam, inspired by the recent development of the second electronic computer (the ENIAC), suggested to John von Neumann that statistical sampling could be used with more efficiency to solve problems In particular, von Neumann outlined a statistical approach to solve the problem of neutron diffusion in fissionable material

The most common application of the Monte Carlo method is Monte Carlo

integration Consider the following integral

This integral represents a non-random problem, but suppose there is a random

vector U, which is uniformly distributed over the region of integration Applying the

function to U, random variable ( ) f U is obtained and the expected value of ( ) f U is

    , where ( ) u is the probability density function of U Being a

pdf, ( ) u will equal 1 over the region of integration and, therefore, we have

   Using substitution,  f U( ) (Monte Carlo Method).

As an example, one can use Monte Carlo integration to find the area under the curve f x( ) on the intervalx3  1,3 This is a problem that can be solved with analyticalmethods and then can be compared to the results when Monte Carlo integration is applied

to approximate the area

The area of the bounded region is given by

For Monte Carlo integration, we will determine the maximum value of the

function on the interval 1,3 This will correspond to the height of the rectangle The width of the rectangle will be 2, the width of the interval over which we are integrating Figure 10 shows the rectangle from which random points will be selected The number

of points that fall under the curve will be counted

Figure 10: Example of Monte Carlo integration.

Monte Carlo integration suggests that

3 3

1 x dx P A 

where P is the proportion of points found below the curve and A is the area of the

rectangular region Using Monte Carlo integration with 10,000 points, the area is 19.845

Of course, the true usefulness of Monte Carlo integration comes in integrating a continuous function that is not integrable by analytical means For example, the normal

curve is given by the function,

2

1 2

1( )

 Consider the standard normal

curve, when   and0  1 This function cannot be integrated The following

demonstrates the use of Monte Carlo integration to estimate

2

1 1

2 0

12

The maximum of g(x) occurs at g(0)  3989 Figure 11 illustrates the integrable

region bounded by a 1 X 1 square

Figure 11: Standard normal curve The square represents the space from which random

points are chosen for Monte Carlo integration

Using 10,000 points generated inside the square,

2

1 1

2 0

12

law of large numbers Suppose there is an event, A , for which one wishes to determine

the probability Let ( ) A be the theoretical probability that event occurs The empirical probability of event A, ( )P A , can be found by designing a controlled experiment that will reduce or eliminate bias Suppose that n trials of the experiment are performed The law

of large numbers implies that as n   , ( ) P A ( )A

One possible source of bias in Monte Carlo methods relates to the random numbergenerator that is used Numbers used in Monte Carlo methods are not truly random but are pseudorandom This means that they use an algorithm of some sort to generate the random numbers A random number generator usually needs a seed value to create a pseudorandom number The random number generator in Java uses the clock for the seedvalue

The following example is presented to illustrate how Monte Carlo methods can beused to estimate

Buffon’s Needle

In 1733, French naturalist Comte de Georges Buffon posed this question:

Suppose a floor is made of parallel strips of wood, each the same width,

and a needle is dropped onto the floor What is the probability that the

needle will lie across a line between two strips?

Buffon presented a solution in 1777 The following is a treatment of the

mathematics involved in that solution

There are two random variables, the angle at which the needle falls ( 0, )

and the distance from the midpoint of the needle to the closest line ( 0,

Figure 12: Illustration of Buffon’s Needle.

Assuming that  and s are independent and uniformly distributed; their joint distribution would also be uniformly distributed Let ( )f x be the probability function for

the random variable  and ( )g x be the probability function for s Then, f x( ) 1



 and

  Let C be the event

that the needle crosses one of the parallel lines, then P(C) can be found by integrating this

joint probability function

sin 2

0 0

2( )

2

1dsd d

Three possibilities exist for this problem:

Case 1: l d (the simplest case)

In this case the ratio 1

dl  ; therefore P(x) = 2

  63662.

Case 2: l d

Here, 1

dl  , which means that P(x) < 2

 , and as l 0 or as d  , then ( )P x  The0probabilities follow the straight line relationship given above by ( )P x

Case 3: l d

This is the most difficult probability to calculate But in general terms, since 1

dl  , this

implies P(x)> 2

 And it follows that asdl  , ( )P x  However after graphing ( )1 P x ,

two problems become apparent

Also, in order for ( )P x to be a valid probability function, the following inequality

must be satisfied: 0P x( ) 1 Therefore, a different rule for ( )P x must be found, when

1

x

Definey lsin When y d , the needle will cross one of the parallel lines Figure 13 shows the feasible region bounded by the rectangle of width and heightd The favorable region, when the needle crosses one of the lines, is shaded Finding the area of this shaded region will reveal the probability of a cross

Figure 13: The feasible region bounded by the rectangle of width and height d

The shaded region can be separated into three regions, two congruent parts and a rectangle Figure 14 shows the rectangle in black surrounded on the left and right by the

two congruent regions The height of the rectangle is obviously d The width of the

rectangle is the difference between the x-coordinates of the intersection of y lsin and

y d This width can be found by subtracting the combined width of the congruent

regions from the width of the feasible region or2sin1 d

 

 l

sin

Figure 14: The area below the curve y lsin bounded by the horizontal lines y d and y is separated into three regions, the rectangle in black surrounded on the left 0and right by two congruent regions.

The area of the two congruent regions can be found by finding the area of the first region and doubling it The area of the first region is

found by integrating the curve y lsin from 0 tosin1 d

d

d d

Combining this result with the previous expression for ( )P x when x1 yields a wise function The function is graphed in Figure 15(Weisstein, Buffon’s Needle Problem and Kunkel, September, 4 2003)

piece-Figure 15: Probability function for Buffon’s needle, where n is the ratio of needle length

to distance between lines

Monte Carlo simulation and this probability function can be utilized to obtain an estimate of Of course, this is not a terribly efficient means of finding , especially if one is actually dropping needles To ensure independence of each trial one would have toremove the dropped needle before dropping the next needle There are many Java appletsonline which allow one to simulate the needle dropping with a great deal more efficiency

An experiment of this nature was conducted by O.C Fox in 1864 while

recovering from wounds sustained during the Civil War His results are shown in the table below (Monte Carlo Method)

n drops m crosses l inches d inches Surface Estimate of 

Table 3

Using an online applet to simulate the dropping of 100,000,000 needles an

estimate of  =3.1415(528912708703) was found, accurate to five digits (Pilana, n.d.)

Results and Mathematics

This paper illustrates a different way to use the Monte Carlo method Starting by

selecting random “points” in a unit n-cube, one counts the number of points that fall inside of a partial n-sphere, centered at the origin Using this proportion one can obtain

an estimate of  by using newly developed functions In two-dimensional space, one actually chooses points from a unit square with its left vertex at the origin and calculates the proportion of points that fall inside of the ¼ circle with center (0, 0) and radius 1

Figure 16: Quarter circle centered at the origin with radius 1 bounded by a unit square.

In three-dimensional space, one chooses points from a unit 3-cube with a vertex atthe origin and three edges along thex x and 1, 2 x -axes Then one must calculate the 3

proportion of points that fall inside the 1/8-sphere In general, this method involves

choosing points from a unit n-cube with a vertex at the origin and with n-edges along the

1, , ,2 n

x x x axes and calculating the proportion of points that fall within the 1

2n n-sphere First, find the volume of each n-sphere, starting with n=2.

The volume of a 2-sphere is This is the area of a circle A 2-sphere is in factR2

a circle and the “volume” is the area A sphere is generated when a semicircle is rotated

about the x-axis Consider the function f x( ) R2x2 rotated about the x-axis The

volume of the sphere is given by

3

R sphere R

Figure 17: Cross-sections of a 4-sphere (Fuquay, 2002)

The disk method can be generalized to any n-sphere if one knows a formula for the volume of a cross-section The volume formula from the (n-1)-sphere is used to

represent the cross-sectional volume Figure 17 shows various cross-sections for the sphere These cross sections would be 3-spheres The volume of a 4-sphere can be