Tài liệu Building OpenSocial Apps- P7 doc

The same rules apply for any piece of data being requested via the OpenSocialframework.The difference is that the performance ramifications are much morenoticeable.Take the case of reque

Managing Your App

Once your app is live, you can hide it, publish changes to the code, or delete it

Hiding and Deleting an App

When you hide an app, you change the status of the application to “Hidden.”Thisonly stops your app from showing up in search results in the apps gallery

(http://apps.myspace.com) It does not hide the app from users who already have the app

installed

Warning

The only way to hide an app from users who already have the app installed or force an uninstall is to delete the app Hiding the app does not hide it from existing users

If you want to delete an app, click Delete.You will be prompted with a confirmation

Click Yes and the app will be deleted

Making Changes to a Live App (Multiple Versions)

Once your app has gone live, there are two versions of it:

n Development version

n Live versionWhen you click Edit App Source on a live app, you see two links: Dev App and LiveApp at the bottom of the screen (as shown in Figure 12.7)

Figure 12.7 Dev App and Live App selection options.

Your Live version can’t be edited directly.You must edit your source code on theDevelopment version Any changes are immediately reflected in the Developmentversion and don’t appear in the Live version until the app is republished.When you’retesting or viewing your Development version, you should see &appvers=devat the end

of the app’s URL, as shown in Figure 12.8 So, if you ever want to see the Developmentversion of your app, just append &appvers=devto the end of the query string for thesurface you’re currently viewing

Republishing a Live App

You’ve changed your code, tested it, made some improvements, and you’re ready to releaseversion 1.1; now what? Click Publish Changes to start the review process over again

Changing the App Profile/Landing Page

When users discover your app in the app gallery or through a friend’s page, they’ll click

on your app link and be taken to your app Profile page.This is like a landing page, and itcontains that magical button that you want everyone to click: Add This App

To see what your app’s Profile looks like, go to the Canvas page and click More Info atthe top of the page Now, do you see a boring default surface with little more than thebasic details? Or something that looks more like the landing page shown in Figure 12.9?

This is one area that developers often overlook, but it’s very important.The app’sProfile is the first introduction of a potential user to your app, so you should try to sellyour app here in order to turn a potential user into a user.You’ll notice that the largeprofessional app development companies have polished-looking app Profile pages

Figure 12.8 Note the &appvers=dev at the end of the URL in the

address bar.

Common tweaks include making the Add This App button larger, adding branding, ting up screen shots of the app in action, and just generally making the page more appeal-ing to prospective installs Editing your app Profile page is similar to the way users “blingout” their MySpace Profile pages, and you’re under the same restrictions.That means noJavaScript, but you can modify CSS and add image, anchor, and various style tags, like bold

put-To edit the app Profile page, you must first log in to MySpace as the app (not as thedeveloper), click Edit Profile, then click the CSS tab Inside the CSS box you can putany valid CSS.You can hook your CSS styles into existing elements on the page usingtheir IDs or element types For example, the Add This App button has an ID of profile_appinstall_button, so you can style it using the #CSS selector Someother useful element IDs are found in Table 12.1, and you can use Firebug or view thesource of the page to see all the markup on the page

Another option is to add custom HTML elements to the About section Inside theEdit Profile page click the About tab; inside the box you can add HTML elementsincluding<img>and<div>tags If any of the tags have a class attribute, you can add theclass definition into the CSS markup under the CSS tab

Here’s an example of how we spiffed up the Tic-Tac-Toe app Profile First, let’s look

at the About section:

<div class="ttt_about_wrap">

<div class="ttt_img">

<img images.myspacecdn.com/images02/3/l_0acdd89ec98c4b7db244ed0ab1371e06.png"/>

src="http://c3.ac-</div>

<div class="ttt_text">

Figure 12.9 App Profile page for iLike (http:/myspace.com/ilike),

a popular application.

#profile_appinstall_button{} Add This App button

#mainLayout{} Main container for the contents of the Profile;

doesn’t include the header or footer of the page or the gray (by default) background

#profile_appdetail{} Container for the top left portion of the Profile,

including the app’s icon, developers, ber of installs, and categories

num-#profile_appinstall{} Container for the top right portion of the

Profile; includes the Add This App button

#profile_aboutMe{} Container for the bottom portion of the Profile,

from “About [app name]” to above the forums

#profile_forums{} Container for the forum links body{} The body element for the page, useful for

modifying the background color and image

padding:10px;

} ttt_img{

float:left;

} ttt_text{

font-family:Verdana;

font-size:18px;

} ttt_text ul{

list-style-position: inside;

text-indent: 20px;

} clear{

Figure 12.10 Updated Tic-Tac-Toe app Profile page.

Managing Developers

We already know how to edit the source code, details, and templates, but you can alsouse the developer platform to manage multiple developers By clicking the Developers &

Testers option under an app on the My Apps page, you can add and delete codevelopers

Once you click on Developers & Testers, you will be shown your friend list.To beadded to your app, the developer must first be your MySpace friend He or she must also

be an approved developer; all of the approved developers in your friend list will have agear icon under their pictures.To add a friend as a developer, click and drag his or herpicture box into the Developers column.To remove the friend as a developer, simplyclick on the red circular icon in the Developers column next to the friend’s name

Before you grant developer status to anyone, it’s recommended that you back up yoursource code with every version of the app

There’s one last kick in the pants As long as you’re someone’s friend and an approveddeveloper, that person can add you as a developer to his or her app and you can’t say no

or remove yourself as a developer.The app creator must remove you So, if someone addsyou as a developer on a questionable app, such as a Battlestar Galactica Sucks app, yourname will be attached to it publicly unless you ask the original creator very nicely toplease remove you as a developer

Managing Testers

You may want to have a few people test your app before you make it live, or test ityourself on different accounts.That’s smart So, how do you do it? From the My Appssection of the developer platform, click the Developers & Testers link.You’ll be shown alist of your friends alongside a Developer column and a Test Users column Simply dragand drop your friends over to the Test Users column to add them as testers Once added,they’ll have instant access to your app in progress

Event Handling—Installs and Uninstalls

The concept of signed data calls was covered extensively in Chapter 8, OAuth andPhoning Home, and it really comes into play here with an app’s Install Callback URLand Uninstall Callback URL.These two settings fields can be found under the EditDetails tab of your app and are shown in Figure 12.11

Basically, whenever a user installs or uninstalls your app, MySpace makes a signedrequest to these URLs and passes the user’s ID So, other than the OAuth signature, it

Suspension and Deletion of Your App

Suspension is pretty rare in the world of MySpace apps If an app violates the Terms ofUse and gets past the reviewers, the app developer usually receives a warning before theapp is suspended or shut down.The developer then has 24 hours (or however long isspecified in the violation notice) to fix the problem.This is not set in stone, and if anapp grossly violates the Terms of Use, or the law, it can be suspended immediately

Any suspension or violation notices are sent to the e-mail of the app’s creator (notthe app’s e-mail address or the e-mail address of any codevelopers), so make sure youmonitor your app developer e-mail address

Warning

If your MySpace account is deleted for violating the MySpace Terms of Use, you will not be able to log in and manage your application Even getting your now-orphaned application deleted can be a major headache There is a work-around for this, though—don’t violate the Terms of Use

If your app is suspended, you need to fix the problem before the app is reinstated So,while you can appeal on the forums, there is no quick fix if your app breaks the rules

Figure 12.11 Callback URL settings.

Summary

If your app works and doesn’t violate the Terms of Use, you should have no problemgetting it published and up and running However, in the rare instance that an app isincorrectly flagged, we hope this chapter will help you navigate the ins and outs of apppublishing

Remember, though, that it’s easier to catch flies with honey, so be courteous whendealing with the developer relations team

Note

Code listings and/or code examples for this chapter can be found on our Google Code page under http://opensocialtictactoe.googlecode.com

ptg

It’s something else entirely to build an app that becomes an everyday part of life for over

a million people Consistent performance and user trust become some of the mostimportant aspects of your app

This chapter is not an exhaustive study of Web app performance If you want to studyWeb app performance, there are many resources online.Yahoo! Developer Network hasdone some excellent research into performance and publishes the information online

You could also pick up any number of books on performance, such as High Performance

Web Sites by Steve Souders It is a large topic worthy of several books.We are, however,

going to cover some of the basics, such as the difference between performance and scaling of an application and some design hints to improve your app’s ability to grow

Performance and Responsiveness

To start this discussion we’re going to explore different aspects of performance

Performance is an overloaded term Cars are often said to have good performance whenthey can reach top speed quickly and handle tight turns without sliding off the road

Another less common but equally correct statement is that a car has good performance if

it gets good gas mileage and can travel long distances on a limited amount of fuel

People are often said to perform well on the job if they are able to handle multiple competing priorities simultaneously It’s not that anything is getting done fast, but thatlots of things are getting done

What Is Responsive Performance and What Is Scale Performance?

Performance is generally discussed in terms of how responsive a system is: “When I clickthe button, does something happen immediately?”To borrow from the auto industry,responsive performance is your “zero-to-60” measure If your app is like my boss’s

Ferrari, it’s doing the equivalent of zero to 60 in 3.5 seconds If your app is more like myjalopy (a 1985 Toyota Tercel, engine factory-rated at 65 HP when it rolled off the assembly line), it’s doing zero to 60 in 35.0 seconds

A secondary, and less common, measure of performance is in terms of scale: “If everyone in the world pushes the button at the same time, does it still work?”This isequivalent to a hot summer day when everyone in the city turns on the air

conditioning If the power grid scales, everyone’s air conditioner continues to functionand homes and apartments cool off If the power grid doesn’t scale, as in California,everyone’s lights flicker or brown out.This behavior is not to be confused with thesituation in the mid-nineties when a rat chewed through a main feeder line, which inturn blew some circuits and took down the Western power grid (a failure of robustness)

Nor is it to be confused with events of the late nineties into the early twenty-firstcentury when Enron was running its “Death Star” and “Fat Boy” projects to createartificial scarcity and brownouts so it could jack up the price and “relieve thecongestion” (a failure of public policy)

Design for Responsiveness

Responsive performance is generally well understood For years the common wisdom inmany computing circles, particularly with Internet applications, was to ignore responsiveperformance and focus on feature delivery.The underlying assumption was that becausecomputing power and network connection speeds were constantly improving, by thetime a given product had reached market penetration, the underlying hardware and network speeds would have improved enough to account for poor performance designs

in the early stages of development

An object example of this idea that hardware would fix all issues with softwareresponsiveness can be witnessed in the early growth and attitudes of the Java community

Programming in Java was very powerful because of the rich and clean abstraction layer itprovided.The promise of a cross-operating-system language that one could “write once,run anywhere” was very compelling Unfortunately for Sun, the hardware performanceenvelope failed to progress quickly enough to account for the initial nonperforming virtual machine design Over the course of a decade the JVM ( Java Virtual Machine) didimprove and hardware did catch up, but the damage was done.Today, because of this and

a number of other reasons, Java is almost never found in user-facing applications and userinterfaces, for which it was originally designed Instead, it is mostly relegated to back-endservers and some embedded applications

While CPU power continues its relentless Moore’s march, overall computer performance has hit a bit of a plateau Much, if not most, of the excess computing power

is being used for visual effects and to support more background processes and level programming tools Another major driver in this performance plateau, in our opinion, is the rise of Web applications and the underlying constraints of network latencies and Web browser rendering engines and JavaScript interpretation

Responsive performance has seen a bit of a renaissance in the last few years Newresearch into user interaction and how great the effect of response times is on user behaviorhas highlighted the importance of responsive design A responsive product is now seen as astrategic differentiator in a world of me-too products.That and new research into what leads

to a responsive design have given rise to a number of rules for writing Web apps

Basic RulesHere are some basic rules that every developer concerned with performance andresponsiveness should know:

n Place CSS styling at the top

n Place JavaScript, particularly includes, at the bottom

n Simplify your markup

At this point in time these rules are bordering on common wisdom, much as “Sizeyour images” and “Make pages smaller” were in the late dot-com days

Responsive OpenSocial App Performance Guidelines

There are also some less obvious guidelines that are specific to writing OpenSocial apps

These rules will help your app be more responsive when dealing with fetching andworking with OpenSocial data Following these guidelines will also help your app perform consistently and keep your users coming back rather than being annoyed byquirks and stutters when using your app

1 Don’t re-request the same piece of data

2 Prefetch record lists for paging

3 Batch requests

4 Handle your errors

We’ll cover each rule in greater detail in the following sections

Rule 1: Don’t Re-Request the Same Piece of DataMany times when we review code, we see this mistake being made Recently we werereviewing some code written in jQuery that was reselecting the same node each time itwanted to change a property:

$("#navhead").addClass("enabled");

$("#navhead").text = "Welcome";

This business went on for another five lines Because it was so convenient to reselectthe node, the author mistook it for being free In actuality each call went through aseven-layer call stack to select the node and add the jQuery extensions.The simple tune-up was to assign the result to a variable and reuse it:

var navHead = $("#navhead");

navHead.addClass("enabled");

navHead.text = "Welcome";

This is fairly basic stuff.We know you jQuery people out there are saying, “Well, thedev should have just chained things.” True, but that is a jQuery-specific feature Stickwith us here for the example

The same rules apply for any piece of data being requested via the OpenSocialframework.The difference is that the performance ramifications are much morenoticeable.Take the case of requesting the Viewer:

In the early versions of the MySpace OpenSocial container, the container code wouldactually make an XHR request to the REST endpoint to refetch the Viewer data everytime this code was called Over time, MySpace has made some good optimizations onthe container side Now, many data call results are either cached and re-served client-side

or prefetched Even with those optimizations, the code still executes a 20-function codecall each time the data is re-requested

The MySpace optimizations for fetched data do not extend to the results ofmakeRequestcalls If your code is calling out to your servers or some other externalserver, only re-execute the call if you think the data might have changed

We’ll start with a simple example.Your app wants to page through the Viewer’s list

of friends, four at a time.The simple and obvious solution is to place code handlersunder the Prev and Next links to create a newFetchPeopleRequest, passing in thestart record and number of records to retrieve.The guts of the code appear somethinglike this:

var startNum = (curPage * pageSize) + 1;

var idspec = opensocial.newIdSpec(

{userId : 'VIEWER', groupId : 'FRIENDS'});

prev.className = "pageaction enabled";

}

}

function loadFriendsPageCallback(data){

var elem = document.getElementById("currentFriends");

var label = document.getElementById("pageHeader");

var fData = data.get("viewerFriendsReq").getData();

var endNum = startNum + pageSize;

label.innerHTML = "Friends " + startNum + " to " + endNum;

loadFriendsPage();

} function getNextFriends(){

Each request shown in the figure is approximately 150ms.That’s not much time inand of itself, but that time adds up If the user goes through six pages, a full second oftime is used waiting for the network.What’s more, if the user pages backward, the samedata your app just had is re-requested.This violates Rule 1: Don’t re-request the samepiece of data If your app is simultaneously attempting to retrieve album information orcommunicate with another server, this design would add noticeable and unnecessaryoverhead

A better design, and one used by the FriendPicker widget, is to ask to preload all thedata in the background, then page by moving through the local data list In order tomake this happen, we use most of the same code with two additional code elements: an

var idspec = opensocial.newIdSpec(

{userId : 'VIEWER', groupId : 'FRIENDS'});

var fData = data.get("viewerFriendsReq").getData();

Figure 13.1 Network traffic on a wasteful pager.

Display of the data is now external to the data retrieval methods and in theshowCurrentPagemethod.This separation of concerns leads to more efficient andmaintainable code.

function showCurrentPage(){

var elem = document.getElementById("currentFriends") var label = document.getElementById("pageHeader")

var startNum = (curPage * pageSize) + 1;

var endNum = startNum + pageSize;

label.innerHTML = "Friends " + startNum + " to " + endNum;

} } elem.innerHTML = items;

var prev = document.getElementById("prevLink");

if(curPage == 0){

prev.className = "pageaction disabled";

}

else{

prev.className = "pageaction enabled";

} }

function getPreviousFriends(){

if(curPage < 1){

return;

} curPage ;

showCurrentPage();

} function getNextFriends(){

The initial request now takes 400ms It happens only once, though.This image alsodisplays all the traffic after scrolling back and forth through the list several times.Thesemeasurements were taken on an account with 52 friends.The step-by-step pager wouldhave taken 1950ms to retrieve the same number of records—almost five times as long!

Not only that, but it would have continued to waste network requests each time the userscrolled.To really see the difference, the user can click the Next link in quick succession

in both implementations.The second displays smoothly and quickly.The first will occasionally stutter, then catch up Sometimes it will skip pages and can even get outsync if the XHR responses are returned out of order

Figure 13.2 Network traffic on a good pager.

Rule 3: Batch RequestsThe OpenSocial spec has a built-in mechanism for batching requests Use it At thetime of this writing, MySpace sends these requests individually, but if you write yourcode to batch things, when the batching performance optimization is made on theunderlying MySpace container, your code will be able to take advantage of itimmediately

For servers under your control, design your endpoints to deliver all the data your appneeds to execute a given task in one shot.This may feel as if it’s violating the integrity ofyour code design, where each type of data gets its own endpoint, but it is a worthwhileoptimization As an example, say your app allows virtual flowers to be sent to a friend

The user has an allotment of credits for buying flowers One design is to have one call topurchase the flowers, then a second call to request the user’s remaining credits andupdate the “available credit” display A better design is to make a single request that purchases the flowers and returns the remaining available credits in the response.You justcut your server request load in half !

Rule 4: Handle ErrorsStability and robustness will be covered in more depth later in this chapter, but they’reworth touching on now Often when an unhandled error occurs in program execution, theprogram is left in an indeterminate state—it doesn’t understand where it is As it attempts

to continue execution, it may make calls with insufficient or invalid inputs, generating evenmore errors More often than one might think, a program descends into a state of thrash—

alternately requesting one piece of data incorrectly, getting a bad response, requesting a different piece of data incorrectly, getting a bad response, and ultimately descending into aninfinite loop, thrashing back and forth between requests until a stack overflow happens orthe browser pegs the CPU and has to be forcefully killed

If this sort of error happens often enough with your app, it may come to the attention of MySpace.When that occurs, you are typically given a few hours to rectifythings If the errors are bad enough that your app is adversely affecting other apps bytying up connections or causing user Home pages to crash, it may even be summarilysuspended Handle your errors

Design for Scale

An application is said to “scale” if it maintains its performance characteristics while thenumber of users increases dramatically MySpace scales At the time of this writing,MySpace has in excess of 150 million unique users Friendster did not scale It predatedMySpace and had more users initially, but once it hit some level of critical mass, thesystem froze and became nonresponsive As a result, everyone left and moved toMySpace

App Guidelines for Internet-Scale Performance

Here are some simple guidelines to consider when designing for scale Follow these rulesand you won’t get tripped up by some of the most common scaling mistakes

1 Know your scaling point

2 Identify scaling bottlenecks

3 Remember what you know

4 Scale horizontally

5 Move beyond relational databases

Application Scale Definitions

Applications are typically designed to perform and scale within one of these definitions:

1 Small/Workgroup Typified by a user base of fewer than 100 users/month Mom-and-pop Web sites, niche e-commerce sites, and internal IT projects designed to address the needs of a single workgroup fall into this range Deployments are usually single-server or partial-server (a server hosting several applications).

2 Enterprise Enterprise-scale applications are designed to satisfy the needs of an entire organiza- tion or a large part of an organization, servicing anywhere from a few hundred to tens

of thousands of users They tend to be industry-specific, like a medical records agement system or a payroll system Deployments may be to a single dedicated server or a small cluster (three to five computers) Classic relational databases and reporting systems are very common Enterprise applications are almost never consumer-oriented.

man-3 Internet Internet-scale applications may service from several hundred thousand to several hundred million users Internet-scale applications are almost always consumer- oriented, although some also provide business-oriented features These applications have unique designs to reach their performance needs The architecture appears foreign to anyone trained solely in classic computer science and data modeling and seems to break many core rules Deployments are across tens, hundreds, or even thousands of servers and must be able to handle individual server failures.

6 Push work to the nodes

7 Consider utility computing

8 Load-test your system

Your app must also scale.What works for a few hundred or even a few thousand usersmay not work for a few hundred thousand users If you do hit the magic tipping pointand your app takes off, there will be little time to redesign things

Take the example of Own Your Friends (OYF), the first big-hit app on MySpace

When Own Your Friends launched, the install rate was just a few hundred a day

Something happened at around 30 days and the app reached the magic tipping point(somewhere between 20,000 and 60,000 installs), and the app install rate jumped by anorder of magnitude At 60 days OYF was growing by over 100,000 installs a day.Thedevelopers (moonlighting at the time) were staying up all night and working weekends

to try to bring more servers online and eliminate bottlenecks.Yes, it may have beenexciting and frantic, but it was also a time that very nearly cost them the business Usersare fickle, and if you fail to execute when your app becomes the big hit, you rarely get asecond chance to make a first impression

Now, let’s delve just a little deeper into our scaling rules

Rule 1: Know Your Scaling PointThe “scaling point” is the point in a system’s utilization where it is no longer able toaccommodate additional usage If this scaling point is outside the system’s expected usageparameters, the current architecture should be adequate If this scaling point is inside thesystem’s usage parameters, you need to adjust your architecture to accommodate a largerscaling point

In terms of pure numbers, we can calculate how much data will overflow the available RAM with some simple math Let’s say you have a user table with the structureshown in Table 13.1

Every row of data is 304 bytes If 2MB are available to hold this database table, yourtable could hold 6578 records and still fit completely in memory.That’s not very manyrecords All this begs the question of how much data is too much for a single server

That’s not an easy question to answer because it depends so much on how your app isusing the data and the overall load of the system

Why don’t more database-backed software applications cause their servers to tip over?

It has to do with partial usage, the principle of locality, and load Irrespective of thenumber of users of a system, not all of them are actively pushing buttons at the same

Table 13.1 Sample User Table

Name 100 Unicode character 200 bytes FavoriteColor 50 Unicode character 100 bytes

use it at a given time.That works out to 100 concurrent users If the average timebetween clicks is 5 seconds, the application must be able to process 20 requests/secondfor this system to operate.

Even with an architecture not specifically designed to scale and the database readingfrom disk on every request, the system should continue to perform handily (the harddisk latency allows for over 100 requests/second).To reach the scaling tipping point ofthis system, usage would have to increase by a factor of 5 (go from 10,000 employees to50,000 employees or increase usage from 10% to 50%)

Rule 2: Identify Scaling BottlenecksThe first step in designing for scale is to identify potential bottlenecks Bottlenecks occurwherever there is a limited resource that many or all of the clients need to access.Thiscould be a REST endpoint on a single server, a routing method, a shared database table,

or even a single shared log file

The first scaling bottleneck most applications hit is the database.This tends to be the

classic point of scaling failure If you can design your system to not use a database, do it

If your needs are simple enough, the app data store may be enough.Then you will bebuilding on the battle-hardened MySpace infrastructure Unfortunately, apps of any com-plexity can rarely get around using a database Other sorts of back-end data stores havebeen experimented with, such as simply writing to a flat file, but a database almostalways outperforms them under load

Dissecting How Hardware Affects the Database

There are several reasons why databases tend to be the primary scaling bottleneck: the physical limitations of reading from a hard drive, CPU saturation from processing queries, and buffer cache running into the wall of available physical memory If a system

is using a misconfigured database, it may also run out of available concurrent database connections To illustrate, let’s examine the read performance of various components in

a computer, from the CPU down to the magnetic hard disk drive Performance tends to change by orders of magnitude In-CPU registers and L1 cache read at a speed of 2 CPU cycles The L3 cache, if present, has a latency of 24 CPU cycles This is incredibly fast.

Hertz is a measure of cycles per second A 1GHz CPU could conceivably find over

1 billion pieces of data from its cache in 1 second

Modern databases use several tricks to speed up performance One of those tricks is

to load the entire database into RAM for processing.This leads to a tremendousimprovement in access speeds—of several orders of magnitude So if the database is inRAM, why is this a scaling performance issue instead of a responsiveness performanceissue? It has to do with database size and how much RAM is available If the database issmall enough to fit entirely in memory, the performance will be off the charts As soon

as the database outgrows the size of physical memory available (about 4GB on a 32-bitmachine, minus the space needed by the operating system), it must store part of its infor-mation on the hard disk If the database has to read from the hard drive into memorytoo many times, the computer starts “thrashing.”Therefore, as soon as your database

Not much data can live in these caches, however The L1 cache in modern CPUs is around 128kB, depending on the processor The L2 cache is commonly 256kB or 512kB Higher-end chips have added an L3 cache as large as 8MB This is still a fairly small amount of data in the grand scheme of things A small database with 5000 data rows could easily fill that cache And the reality is that CPU cache is going to be filled with numbers and instructions and bits the computer needs to execute its immediate responsibility There’s no room for your data there, other than what is being actively worked on.

Moving from cache memory that is directly on the CPU to RAM memory, the performance jumps by another order of magnitude The L3 cache access of 24 CPU cycles is now closer to 200 CPU cycles on 6ns RAM That translates to theoretically finding 5 million pieces of data in a second using a 1GHz CPU This is still nothing to scoff at, but it’s quite a difference from data on the CPU cache.

A hard disk, where most of your data lives, is not one, not two, not three, but several orders of magnitude slower This calculation is not completely accurate since it depends on CPU frequency and bus speed, but it is about 1 million times slower to read data off a hard drive than it is to read it out of memory A 7200RPM drive has a latency of approximately 4ms and a seek time of 9ms This correlates to finding somewhere between 100 and 250 distinct pieces of data from hard disk in

1 second.

1ms = 1,000,000 ns

On average, in the time it takes to find one piece of data on hard disk, a computer could find 20,000 pieces of data in RAM Of course, these numbers are somewhat deceptive since they are calculated based solely on latency and seek times In reality, the data should be stored contiguously and read from the seek point Then the constraints become hard disk platter rotation speed and bus transfer rate The numbers we’ve given are enough to illustrate the point, however.

This performance spread is why most databases attempt to load all available data into RAM memory There was a tremendous performance boost to SQL Server once it started loading entire tables of data off hard disk and completely into memory In this manner an entire database can be loaded into RAM and processed at a rate several orders of magnitude faster than if the physical hard drive had to be accessed continuously.

crosses that line in terms of the amount of data stored, the performance of your app willplummet Code that previously could execute immediately will become blocked waitingfor the database to respond.This in turn ties up HTTP and database connections.Theconnections will begin to queue up until the queue is full Any new connections getturned away, and the Web server returns 503 “Server too busy” errors At MySpace, werefer to a server exhibiting this behavior as “tipping over.”

Rule 3: Remember What You KnowThis rule is a variation of “Don’t re-request the same piece of data” from the responsiveperformance section Let’s say we are going to see the kind of growth outlined previously and expect usage to jump from 10% to 80%—well above our parameters.Thefirst step is to move often accessed and seldom changing data into a location where it isguaranteed to be in RAM For example, let’s assume the app in question allows users topick their favorite color.The 20 or so available colors and their hex definitions should be

loaded from the database once and placed into a static hash table object.This avoids any

question of whether the data will be read from memory or disk.The same holds true forall the domain reference tables—lookup data defined in the system that rarely or neverchanges

Much of the data being accessed is read-only or limited to a certain session Oftenthis can be placed in a caching layer, like a MySQL MEMORY table or a memcachedsystem distributed across a cluster of computers.These systems start to get fairly complicated and are a little beyond the scope of what this book covers, but be aware thatthey exist In reality, a reasonably intelligent database, like SQL Server, does a fairly goodjob of automatically keeping the most frequently accessed records in memory, providedthe number of “most recently used” records can fit in a single system’s memory

Rule 4: Scale HorizontallyOnce an app hits a point where it can no longer live on a single computer, it must besplit to scale across multiple computers.There are two ways to do this: vertically and horizontally.Vertical scaling is scaling by specialty: One computer is the Web server andanother is the database Another might serve static files, like a content delivery network(CDN) Horizontal scaling is distributing the same work across many computers.TheURL www.example.com could resolve to a Web farm of hundreds of machines.Thedatabase can be federated across many, many computers Each computer has the samekind of data but is responsible for only a small segment of the total data, say, 500,000records

Vertical scaling is typically used to improve responsive performance Long-running orexpensive processes, like running reports, could easily bog down a Web server whileservicing only a few users Unfortunately, there is a limit to how much specialized processing can be segmented out, so vertically scaled systems often top out at distributingthe load across two to five machines (Web server, CDN, transactional database server,reporting database server, data warehouse) Some things simply can’t be vertically scaled,

such as a system containing 100 million user records Horizontal scaling can distributeload across a much larger set of machines Hundreds of machines are sometimes used toscale out different feature areas of MySpace

Designing for horizontal scale means not requiring all of your data to exist on thesame machine or to come from any other single machine Classic ASP Session handlingwas notorious for causing performance problems and hampering horizontal scaling

The problems were actually twofold: It dropped the Web server into single-threadedexecution per user, and it couldn’t be distributed across different computers In usingclassic ASP Session, an application essentially condemned itself to not using a Web farmand not being able to scale

Rule 5: Move Beyond Relational DatabasesRelational databases are hard to beat within their domain space—small to enterprise-class transactional systems Joining data tables starts to lose its efficiency once a system has

to scale across multiple servers Large federated database systems often perform betterwhen laid out in a dimensional model, like a data warehouse.There are data redundancyissues, but the performance gains of avoiding multiple joins can be worth it

Rule 6: Push Work Out to the Nodes

In the same way that using a Web farm distributes request load across many computers,some work can be pushed all the way down to the app running in a client browser Byusing the JavaScript engine in client browsers, your app can create what amounts to agiant grid computer.Whenever practical, perform calculations in the client script prior

to sending the information back to the server Consider an app that needs to draw a piechart for each user based on the user’s data It could be done efficiently for a few users

on the server, then pushed to the client as an image.Thousands of requests would makethis operation a bottleneck.This work would scale better if the data points were pushed

to the client and the browser drew the pie chart using Canvas, Flash, or any of a number

of JavaScript graphing solutions, such as PlotKit

Rule 7: Consider Utility ComputingUtility computing may have sounded like crazy talk a few years ago, but today it makessense Amazon Web Services, Google App Engine, and Microsoft Azure are the three bigplayers in this space Rather than paying your own network engineers and leasing rackspace, it can be much more cost-effective to build on one of these services and leveragethe expertise of a multibillion-dollar corporation

Utility computing, by its very nature, is designed for Internet-scale businesses One ofthe reasons utility computing sounded like snake oil a few years ago was that fewapplications needed Internet scale; most were only enterprise scale Existing applicationswere also not designed to take advantage of a utility computing architecture, and anexisting application is unlikely to port easily to a utility computing model