Tài liệu MySQL Administrator’s Bible- P14 docx

We can use the INFORMATION_SCHEMAtableSTATISTICSto show all the indexes on therentaltable: mysql> SELECT COLUMN_NAME, INDEX_NAME, SEQ_IN_INDEX AS pos -> FROM INFORMATION_SCHEMA.STATISTIC

Trang 1

Query Analysis and Index Tuning 18

The reason this is a full table scan is that there are no suitable indexes to use We can use the

INFORMATION_SCHEMAtableSTATISTICSto show all the indexes on therentaltable:

mysql> SELECT COLUMN_NAME, INDEX_NAME, SEQ_IN_INDEX AS pos -> FROM INFORMATION_SCHEMA.STATISTICS

-> WHERE TABLE_SCHEMA=’sakila’ AND TABLE_NAME=’rental’;

7 rows in set (0.11 sec)

There is no index that includes thereturn_datefield, so add an index to optimize this query:

mysql> USE sakila;

Database changed mysql> ALTER TABLE rental ADD INDEX (return_date);

Query OK, 16044 rows affected (12.08 sec) Records: 16044 Duplicates: 0 Warnings: 0

mysql> EXPLAIN SELECT return_date FROM rental\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE table: rental type: index possible_keys: NULL

key: return_date key_len: 9

ref: NULL rows: 16249 Extra: Using index

1 row in set (0.00 sec)

Now thetypeisindex, which means a full scan of an index is being done The index beingscanned is thereturn_dateindex (key), which we just created, with a length (key_len) of 9

Is there a way to further optimize this query?

Looking at Table 18-1, data access strategy types belowindexinvolve using only parts of anindex The query we are analyzing returns every value of thereturn_datefield Therefore,there is no way to avoid accessing every value in thereturn_dateindex.mysqldneeds to

Trang 2

Part IV Extending Your Skills

access a value in order to return it, and every value is returned, so every value must be accessed.This need to access every value is also shown by the lack of Using wherein theExtrafield

Index consequences

In Chapter 6, we explained how indexes work Indexes can make data retrieval faster becausethey are ordered subsets of data, and can be searched faster than the entire set of data, whichmay be ordered differently than an index There is a cost to maintaining indexes Data changes areslower because the data needs to be inserted into the table and any appropriate indexes need to beupdated An index needs uses disk space, memory, and processing power to stay up to date

When analyzing queries, remember that there are tradeoffs for actions Many times, adding an indexwill make an application run faster because the query runs faster However, there are times whenadding an index makes an application run more slowly, because although theSELECTquery runsfaster, theINSERT,UPDATE, andDELETEqueries run more slowly

It helps to be familiar with the nature of all the queries against the database If you find that selecting

a field from a table that stores user session information is slow, adding an index may make theapplication slower because there are many changes to user session information From time to time,you may want to reexamine indexes to ensure that they are being used An index that is not beingused is a waste of resources

Optimizing away Using filesort

TheExtravalueUsing filesortis not desirable; it means thatmysqldhas to pass throughthe data an extra time in order to sort it If theExtravalueUsing filesortshows up in asubquery, it is best to optimize the query by eliminating the subquery In queries that do notinvolve subqueries, theExtravalueUsing filesortmay occur in theEXPLAINplan forqueries that useORDER BY,DISTINCT, andGROUP BY

ON the WEBSITE

ON the WEBSITE More information on how to create and use subqueries can be found on the accompanying website for this book at www.wiley.com/go/mysqladminbible.

For example, the followingEXPLAINplan is for a query to find the customer name and activestatus based on an e-mail lookup, sorted by last name:

mysql> EXPLAIN SELECT first_name, last_name, active -> FROM customer WHERE email=’barbara.jones@sakilacustomer.org’ -> ORDER BY last_name\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE

618

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.

Trang 3

table: customer type: ALL possible_keys: NULL

key: NULL key_len: NULL ref: NULL rows: 541 Extra: Using where; Using filesort

In order to optimize away theUsing filesort, you need to have an index that

mysqldcan use instead of sorting In most cases,mysqldcan only use one index, soyou will need to have an index that handles both the sorting and the filter of WHERE email=’barbara.jones@sakilacustomer.org’:

mysql> ALTER TABLE customer ADD INDEX (email, last_name);

mysql> EXPLAIN SELECT first_name, last_name, active -> FROM customer WHERE email=’barbara.jones@sakilacustomer.org’

-> ORDER BY last_name\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE table: customer type: ref possible_keys: email

key: email key_len: 203 ref: const rows: 1 Extra: Using index condition; Using where

You have removed the undesirableExtravalueUsing filesort, and added the desirable

Using index condition You have also gone from a data access strategy (type) of full tablescan (ALL) to one of looking up a nonunique index value (ref)

Often, first instincts may not fully optimize a query For example, your first instinct in mizing this query might have been to add an index on only theemailfield This would haveoptimized the data access strategy, but the query would still have anExtravalue ofUsing filesort Having one index for both fields allowsmysqldto use that index to optimizethe data access strategy and the filesort It is always a good idea to test as many optimizationsolutions as possible see the sidebar ‘‘Testing ideas.’’

Trang 4

opti-Part IV Extending Your Skills

Testing ideas

In the example from the section ‘‘Optimizing away Using filesort,’’ you might have tried to see if

mysqldwould use an index onlast_nameonly; if that was your first instinct, you can try outthe following commands to see if the index would work:

ALTER TABLE customer DROP KEY email;

ALTER TABLE customer ADD INDEX (last_name);

EXPLAIN SELECT first_name, last_name, active FROM customer WHERE email=’barbara.jones@sakilacustomer.org’

ORDER BY last_name\G

Sometimes, the first idea you have to optimize a query will not actually optimize the query In thiscase, the index onlast_namedoes not help becausemysqldneeds to filter for theWHEREclausefirst, before ordering Ifmysqldwas to use the index onlast_name, it would have to go throughthe entire index, and for each row in the index, look up theemailfield from the data to see if itmatched If there were a match, thelast_namewould be put in the result set, and thefirst_name

andactivefield would be looked up and also put in the result set Those lookups are a lot of extrawork, and the query optimizer rightfully uses a full table scan, even with an index onlast_name.There will be other times when the best solution for optimization is not the best solution overall forthe application In this example, an index was added on (email,last_name) and theEXPLAIN

plan showed a key length (key_len) of 203 That is a very large key to keep up to date, and if itslows down the application, it may be more beneficial to use an index with a shorter length, even

if it meansmysqldhas to do afilesort

Optimizing away Range checked for each record

As shown in Table 18-2, theExtravalueRange checked for each recordis faster than

a full table scan (type: ALL) but slower than a full index scan (type: index) To optimizequeries with thisExtravalue, create or modify an index so that the query optimizer has a goodindex to use Often, optimizing queries to get rid of Range checked for each recordresults

in a data access strategy (type) ofrange,reforeq_ref

Optimizing away Using temporary

Unlike in previous discussions, optimizing away anExtravalue ofUsing temporarycannot

be done by adding an index.Using temporaryis undesirable, as it means that a temporarytable must be used to store intermediate results There are several ways to optimize this,depending on why a temporary table is used:

■ IfORDER BYandGROUP BYare both present, and use different fields and/or ordering,the way to optimize this is to get rid of either theORDER BYor theGROUP BY This may

620

Trang 5

be done by splitting the query into two queries It may be possible to combine the twoqueries by usingUNIONso that intermediate results do not need to be stored in a tempo-rary table

■ The presence ofORDER BYandDISTINCTmay cause a temporary table to beused The way to optimize this is to get rid of either theORDER BYor theDISTINCT Thismay be done by splitting the query into two queries It may be possible to combine thetwo queries by usingUNIONso that intermediate results do not need to be stored in atemporary table

■ If theSQL_CALC_FOUND_ROWSkeyword is used, the number of rows is stored in a porary table, which can be retrieved by issuingSELECT FOUND ROWS() To optimize, getrid ofSQL_CALC_FOUND_ROWS Depending on what you are counting, you might countresults periodically and have an estimate for a time period (i.e., run a query every 10 min-utes to put the number into table and read the table, doing one count every 10 minutesinstead of one count every time the query is issued)

tem-■ TheSQL_SMALL_RESULTkeyword is used in aSELECTstatement withDISTINCTor

GROUP BY TheSQL_SMALL_RESULTkeyword is a hint to the optimizer that the result

is small, and thus it should use a temporary table instead of a filesort To optimize, get rid

ofSQL_SMALL_RESULT If you need theSQL_SMALL_RESULTkeyword because a rary table is more desirable than a filesort, then you cannot optimizeUsing temporary

tempo-away

If you use optimizer hints, be sure to run periodic testing Only through periodic ing can you determine whether a temporary table or a filesort is better for your particularsituation

test-■ ORDER BYorGROUP BYis used on a field that is not the first table in the join queue (thefirst row returned in theEXPLAINplan) One way to optimize this query is to change oreliminate theORDER BYclause Another way would be to change the filter so that the tableorder changes

For example, the following query uses thecustomertable first in the join queue, but issorting based onrental_date, a field in therentaltable:

mysql> EXPLAIN SELECT first_name, last_name FROM rental -> INNER JOIN customer USING (customer_id)

-> ORDER BY rental_date\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE table: customer type: ALL possible_keys: PRIMARY

key: NULL key_len: NULL ref: NULL rows: 591 Extra: Using temporary; Using filesort

Trang 6

*************************** 2 row ***************************

id: 1 select_type: SIMPLE table: rental type: ref possible_keys: idx_fk_customer_id

key: idx_fk_customer_id key_len: 2

ref: sakila.customer.customer_id rows: 13

Extra:

To optimize this query, we could change theORDER BYto use a field in the customer table,

or we could change the query to use the rental table first in the join queue Join tableorder can be forced by using a join type ofSTRAIGHT_JOIN(which cannot use theUSING

syntax):

mysql> EXPLAIN SELECT first_name, last_name FROM rental -> STRAIGHT_JOIN customer ON rental.customer_id=customer customer_id

-> ORDER BY rental_date\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE table: rental type: index possible_keys: idx_fk_customer_id

key: rental_date key_len: 13

ref: NULL rows: 16291 Extra: Using index

*************************** 2 row ***************************

id: 1 select_type: SIMPLE table: customer type: eq_ref possible_keys: PRIMARY

key: PRIMARY key_len: 2 ref: sakila.rental.customer_id rows: 1

Extra:

However, this may or may not actually make the query better —Using filesortisgone, but the data access strategy for the rental table is much slower In general, usingtechniques like index hints andSTRAIGHT_JOINare dangerous query optimization

622

Trang 7

strategies, because changes in the amount of data, the cardinality of data, and the schemamay change the optimal query plan If you must use these techniques, reassess theirvalidity every few months and whenever complaints of database slowness arise

A better way to change the order of the join queue is to limit the rows examined in thedesired table For example, you can limit the rows examined inrentaltable to a certainrange:

mysql> EXPLAIN SELECT first_name, last_name FROM rental -> INNER JOIN customer USING (customer_id)

-> WHERE rental_date BETWEEN ’2005-01-01 00:00:00’ AND -> ’2005-01-31 00:00:00’ ORDER BY rental_date\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE table: rental type: range possible_keys: rental_date,idx_fk_customer_id

key: rental_date key_len: 8

ref: NULL rows: 1 Extra: Using where; Using index

*************************** 2 row ***************************

id: 1 select_type: SIMPLE table: customer type: eq_ref possible_keys: PRIMARY

key: PRIMARY key_len: 2 ref: sakila.rental.customer_id rows: 1

Extra:

It is beneficial to optimize awayUsing temporarybecause in certain cases, temporary tableswill be written to disk These situations include: when a temporary table exceeds the smaller

oftmp_table_sizeandmax_heap_table_size, when a temporary table includesBLOBor

TEXTdata types, whenDISTINCTorGROUP BYclauses contain fields that use more than 512bytes, and when any field is more than 512 bytes in aUNIONorUNION ALLquery

Using an index by eliminating functions

Sometimes, an index exists but is not being used For example, thefilmtable has the followingindexes:

mysql> SELECT COLUMN_NAME, INDEX_NAME, SEQ_IN_INDEX AS pos -> FROM INFORMATION_SCHEMA.STATISTICS

-> WHERE TABLE_SCHEMA=’sakila’ AND TABLE_NAME=’film’;

Trang 8

However, the following query does not use the index ontitle, as you might expect it would:

mysql> EXPLAIN SELECT title FROM film WHERE LEFT(title,2)=’Tr’\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE table: film type: ALL possible_keys: NULL

key: NULL key_len: NULL ref: NULL rows: 953 Extra: Using where

The reason for this is that there is an index ontitle, but theWHEREclause is filtering based

on a function of thetitlefield Values (such as’Tr’) cannot be compared to a function(LEFT(title,2)) using an index inmysqld, unless the index is on the function itself.Unfortunately,mysqlddoes not support an index on functions, and so it is not possible todefine an index onLEFT(title,2)even if you had the desire

To optimize this type of query, see if you can take away the function In this case, you canreplaceLEFT(title,2)=’Tr’withtitle LIKE ’Tr%’to get rid of the function ontitle.Just by changing the query to get rid of the function, you can change your data access strategyfrom atypeofALLto atypeof range:

mysql> EXPLAIN SELECT title FROM film WHERE title LIKE ’Tr%’\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE table: film type: range possible_keys: idx_title

key: idx_title key_len: 766 ref: NULL rows: 15 Extra: Using where

624

Trang 9

This type of optimization is done most frequently to queries involving date ranges

However, there are other ways in which functions can be optimized out of a query Table 18-4shows some common optimizations:

TABLE 18-4

Common Ways to Optimize by Eliminating Functions

WHERE clause Function Optimization

DATE(datefield) = ’2005-05-30’ or LAST_DAY(field)=’2005-05-30’ or LEFT(datefield, 10) = ’2005-05-30’

SUBSTRING_INDEX(datefield,’ ’) =

’2005-05-30’

field BETWEEN ’2005-05-30 00:00:00’ AND ’2005-05-30 23:59:59’

CONCAT(field,’day’)=’Saturday’ field =’Satur’

FROM_UNIXTIME(field)=’2005-05-30 00:00:00’

field= 1117425600

LEFT(INET_NTOA(field),10)=’192.168.1.’ field BETWEEN 3232235777 AND

3232236031

You may be wondering why anyone would ever createWHEREclauses like the ones in Table

18-4 Most of the time it happens because of the way the developer is thinking Developers writequeries to answer questions, so these types ofWHEREclauses happen when the developer writes

a query to ‘‘find sales on May 30’’ or to ‘‘find distances greater than 20’’ In an ideal world, noquery would be saved to code unless it were optimized In practice, developers write queries,

Trang 10

and DBAs optimize queries — if the developer writes a suboptimal query, in many organizationsthe DBA will find it only when it slows down the application

Optimizing the last two queries in Table 18-4 requires some work to retrieve the numerical ues To optimizeFROM_UNIXTIME(field)=’2005-05-30 00:00:00’, you have to find the

val-UNIXtimestamp value for the datetime There is a function to do that:

mysql> SELECT UNIX_TIMESTAMP(’2005-05-30 00:00:00’);

+ -+

| UNIX_TIMESTAMP(’2005-05-30 00:00:00’) | + -+

+ -+

To optimizeLEFT(INET_NTOA(field),10)=’192.168.1.’, you first have to figure out whatthe query is looking for This filter finds rows that havefieldwith the numerical equivalent

of an IP address whose left 10 characters are’192.168.1.’ Another way to look at the ter is that it finds rows that havefieldwith the numerical equivalent of an IP address between

fil-192.168.1.1and192.168.1.255

This new way to look at the data presents you with a way to eliminate the function from the

WHEREclause If you find the numerical equivalent of the boundary IPs, you can use those intheBETWEENcomparison shown in Table 18-4 Again,mysqldhas a function that will let youlook those values up:

mysql> select INET_ATON(’192.168.1.1’), INET_ATON(’192.168.1.255’); + -+ -+

| INET_ATON(’192.168.1.1’) | INET_ATON(’192.168.1.255’) | + -+ -+

+ -+ -+

There are functions that simply cannot be eliminated For example, it is difficult to eliminate

WHEREclauses such asMOD(field,10)=2andLENGTH(field)<5

Non-index schema changes

Sometimes the best way to optimize a query is to change the data structure Consider the ing query:

follow-mysql> EXPLAIN SELECT first_name, last_name, email -> FROM staff

-> WHERE email LIKE ’%sakilastaff.com’\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE

626

Trang 11

table: staff type: ALL possible_keys: NULL

key: NULL key_len: NULL ref: NULL rows: 1 Extra: Using where

An index onstaffwould not help, because text indexes work from the beginning of the word(see Chapter 6) Much like a dictionary, an index is set up to look for words that begin a cer-tain way, not words that end a certain way The way to optimize this query would be to have anindex that could look up words that end a certain way

Such an index does not exist However, what you can do is add a field to the table that containsthe text reversed, with a regular index on that For example:

mysql> ALTER TABLE staff ADD COLUMN revemail VARCHAR(50) DEFAULT

NULL, -> ADD INDEX (revemail);

mysql> UPDATE staff SET revemail=REVERSE(email);

Query OK, 2 rows affected (0.08 sec) Rows matched: 2 Changed: 2 Warnings: 0

mysql> SELECT email, revemail FROM staff;

2 rows in set (0.03 sec) mysql> EXPLAIN SELECT first_name, last_name, email -> FROM staff

-> WHERE email LIKE ’%sakilastaff.com’\G

You can use theREVERSE()function to show you what the comparison string should be, andthen run anEXPLAINto see if the new field and index help:

mysql> SELECT REVERSE(’%sakilastaff.com’);

+ -+

| REVERSE(’%sakilastaff.com’) | + -+

+ -+

Trang 12

mysql> EXPLAIN SELECT first_name, last_name, email -> FROM staff

-> WHERE revemail LIKE ’moc.ffatsalikas%’\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE table: staff type: range possible_keys: revemail

key: revemail key_len: 203 ref: NULL rows: 1 Extra: Using index condition; Using MRR

You have optimized the query to have a data access strategy (type) of range You also have thedesirableExtravaluesUsing index conditionandUsing MRR— see Table 18-2 for details.Note that having an extra field also means upkeep of the field The best way to upkeep the fieldwould be to have any code that inserts or updates theemailfield also update the newreve- mailfield Another option is to create triggers to update therevemailfield when theemail

field is changed See Chapter 7 for more information about triggers

Batching expensive operations

INSERTandDELETEoperations need to update indexes, andDELETEoperations can cause mentation These operations are particularly expensive on MyISAM tables, as the table blocksall other reads and writes until theINSERTandDELETEoperations are complete One way tooptimizeINSERTqueries on MyISAM is to use the system variableconcurrent_insert, whichallows some nonblockingINSERToperations Another way to optimize these queries, regardless

frag-of storage engine, is to batch expensiveINSERTandDELETEoperations

INSERToperations can be batched by using theINSERT DELAYEDsyntax For more information

onINSERT DELAYED, see Chapter 4

To batchDELETEoperations and reduce fragmentation, use a table for deleted records:

CREATE TABLE deleted_customer ( customer_id smallint(5) unsigned NOT NULL PRIMARY KEY ENGINE=InnoDB DEFAULT CHARSET=utf8;

When a customer is deleted, they are flagged for deletion by inserting theircustomer_idintothedeleted_customertable When a query looks for nondeleted customers, it can use a veryfast join with the desirableNot existsvalue forExtra:

mysql> EXPLAIN SELECT first_name, last_name -> FROM customer LEFT JOIN deleted_customer USING (customer_id) -> WHERE deleted_customer.customer_id IS NULL\G

628

Trang 13

*************************** 1 row ***************************

id: 1 select_type: SIMPLE table: customer type: ALL possible_keys: NULL

key: NULL key_len: NULL ref: NULL rows: 579 Extra:

*************************** 2 row ***************************

id: 1 select_type: SIMPLE table: deleted_customer type: eq_ref

possible_keys: PRIMARY

key: PRIMARY key_len: 2 ref: sakila.customer.customer_id rows: 1

Extra: Using where; Using index; Not exists

BecauseDELETEoperations are expensive, a batchDELETEshould be scheduled for nonpeaktimes After deleting rows from the customer table, defragment the table Table defragmentationwill lock the table with a write lock, blocking all reads and writes, but that is another reason toperform batch deletions during nonpeak times The batch deletion is done by running:

DELETE customer.* FROM customer INNER JOIN deleted_customer USING (customer_id);

TRUNCATE TABLE deleted_customer;

OPTIMIZE TABLE;

Even though the table join when retrieving nondeleted customer information is fast, it is moreoverhead However, because expensiveDELETEoperations are batched, overall the applicationwill run much faster if problems are caused by having manyDELETEoperations

Optimizing frequent operations

There may be a database query that is run extremely frequently Even if it is not a particularlyslow or expensive query, it is contributing to load on your servers Example queries retrieveinformation such as:

■ The number of currently online customers

■ The total number of active customers

Trang 14

■ Rankings

■ Tag clouds

■ How many articles or other resources appear on the site

■ How many comments an article has

■ Up-to-the-second calculations of account balancesThese queries can often be batched For example, do your customers need the exactup-to-the-second number of logged-in customers? This may be a number displayed on everyweb page, including pre-login pages, to get people to log in and stay logged in It may besufficient in these cases to calculate the number of logged-in customers once every 5 minutes.Depending on the usage of your application, calculating rankings, number of comments on anarticle, and account balances may make more sense to be denormalized instead of calculated onthe fly For example, the following query lists customers in order from those who have paid themost to those who have paid the least:

mysql> EXPLAIN SELECT first_name, last_name, SUM(amount) AS

total_paid -> FROM customer INNER JOIN payment USING (customer_id) -> GROUP BY customer_id ORDER BY total_paid DESC\G

*************************** 1 row ***************************

id: 1 select_type: SIMPLE table: customer type: ALL possible_keys: PRIMARY

key: NULL key_len: NULL ref: NULL rows: 591 Extra: Using temporary; Using filesort

*************************** 2 row ***************************

id: 1 select_type: SIMPLE table: payment type: ref possible_keys: idx_fk_customer_id

key: idx_fk_customer_id key_len: 2

ref: sakila.customer.customer_id rows: 14

Extra:

If this query is run every time a report is accessed, it may be running quite often, depending onwho can access the reports! Rankings, number of comments per article, and tag cloud calcula-tions are frequently run at a rate higher than the data actually changes

630

Trang 15

One way to optimize reporting queries and other calculations that are run more frequently thanthe data changes is to denormalize the data, using new field(s) or table(s) to store the relevantcalculations When the data is changed, the denormalized field(s) must also be updated Thiscan be done by changing your application code or by adding a trigger For more on triggers, seeChapter 7

Another way to optimize these types of queries is to run them periodically, storing the results

in the database, file, or globally accessible application-wide variable This can pose problems

if users are expected to seeing up-to-the-second current results, so try to train your customers

to expect recent, but not necessarily up-to-the-second, results If there is resistance to changinguser expectations, calculating features such as rankings periodically can be a great marketingtool Customers will likely log in more often if their rankings change every 4–6 hours, instead

of immediately after each action they take

Summary

In this chapter, you have learned how to useEXPLAINto generate a query plan How data anddata changes over time can affect the query plans has been discussed You have explored how tomake a query plan better by:

■ Adding indexes

■ Changing the size and type of data fields

■ Adding new data fields

■ Moving indexed fields out of functions

■ Limiting the use of temporary tables

■ Batching expensive and/or frequent queries

■ Periodically calculating frequent queries

Trang 17

Monitoring Your Systems

IN THIS CHAPTER

Deciding what to monitor Examining open source monitoring

Examining commercial monitoring

In Chapter 17, we discussed how to measure database performance

By taking regular measurements of the health of your systems, youcan learn about database problems before they are happening, send amessage as they are happening, and have information to help debug afterthey happen This type of automated monitoring can make your job eas-ier, whether monitoring alerts you when a system is down, or gives youthe information necessary for capacity planning

It is surprising how many organizations do not have any automatedalerting, because ‘‘when the database is down, we know because users callus.’’ Alerting frameworks are easy to set up and use and can let you knownot only about current problems, but alert you to what may be a problem

in the future For example, a full disk will causemysqldto hang andmay cause corrupt data and log files Alerting when a disk is approachingcapacity can save a lot of problems — you can clear out some space andmake sure the disk does not become full

Sending an alert is as easy as sending an e-mail; most pagers and cellphones have e-mail addresses that can be set up to send the device amessage It may be tempting to schedule scripts that, when they fail, send

an e-mail However, there is a lot more flexibility in alerting systems that

do not take much time to set up Alerting systems come bundled withbasic checks, and the best ones are ones where you can write your ownchecks and easily integrate them into the alerting framework

Even in organizations that have alerting, graphing trends and patterns ends

up being a low priority However, organizations that graph data are betterable to handle alerts, find the root causes of problems, and predict whenfuture problems may occur Though graphs are not necessarily examineddaily, they are an immense help when they are examined

Trang 18

It should be pointed out when examining the output of whatever program that you use for monitoring that, as an example, just because your systems are operating at an average of 10% of resource utilization that you expect the systems to be able to operate with 10 times that load The reason why is that systems scaling is never a linear issue When you hit 20% utilization you might uncover a bottleneck that keeps your systems from working well until you resolve the problem.

We will discuss some of the widely-used open source monitoring tools, as well as the MySQLEnterprise Monitor that is part of paid support for MySQL, and MONyog, another commerciallyavailable monitoring tool While all of the solutions presented in this chapter are suitable forproduction use in an enterprise environment, you may find that some are more suitable thanothers for your environment For example, many of the tools depend heavily on SNMP (theSimple Network Management Protocol), which some organizations deem unsecure Otherorganizations may require an agent-less protocol, or that the monitoring tool(s) run on theWindows operating system The many solutions provided in this chapter reflect the needs ofdifferent organizations

Deciding What to Monitor

Monitoring, like documentation, is an ongoing process New features inmysqldand changes indatabase use may require changing what is monitored Monitoring adds overhead to your sys-

tems and takes time to implement and maintain, so monitor everything is not always desirable.

Exactly what you should monitor is different for different environments Each item you monitormust have a purpose for monitoring — for example, you may monitor replication lag to be able

to see trends in how far behind replication lags You may also monitor replication lag to alertyou when replication lags beyond a certain point, as that may indicate a bigger problem.Thinking of the reasons for monitoring can help you figure out what you need to monitor Somereasons for monitoring are:

■ Alerting when:

■ The database is unavailable

■ A database error occurs

■ InnoDB tablespaces are running out of space

■ The file system is running out of space (mysqldhangs when it cannot write to a log

or data file)

■ The file system is running out of memory

■ A query has been running for a long time

■ There are many connections, approaching max_connections

■ The database response time is unacceptable

634

Trang 19

Monitoring Your Systems 19

■ Replication is lagging behind

■ Temporary disk tables get very large

■ Slow query rate is high

■ Number of sleeping queries is high

■ Graphing and seeing trends/patterns for:

■ Database/system uptime

■ InnoDB buffer pool efficiency

■ Query cache efficiency

■ How much disk space the database uses

■ How many users your application has

■ Table locks

■ How often tables in a database are changed

■ File I/O caused by the database

■ Number and size of reads/writes/transactions

■ Replication lag

■ Table cache efficiency

■ Number of connections and their states (sleeping, copying to temp table, query)

■ Frequency of query joins, sorts, full table scans, range scans

■ Temporary table and temporary disk table usage

This list is not exhaustive, but it is a good start to thinking about what you might want to tor Note that the items on the list above point to more than simply monitoring database param-eters The best monitoring solutions take into account the database, the server, the application,and other architectural pieces of the system such as load balancers and switches

moni-Many organizations run an excellent alerting solution (such as Nagios) and an excellent graphingsolution (such as Cacti), though this has the overhead and duplication that information aboutwhat hosts and services to monitor is in more than one place However, the alternative is anall-in-one solution that is not as strong in both alerting and graphing

When deciding what to use for your monitoring needs, consider how the organization as awhole can benefit from your choices It is not uncommon for a DBA to install monitoring

on their systems, and have a systems administrator say ‘‘can you just monitor Apache on thewebservers, too?’’ Small projects can grow, and with the right technologies in place, you canstart the ball rolling on a monitoring system that will be able to see details such as whether ornot a server is running, as well as overall pictures such as how many concurrent users yourapplication can withstand

Monitoring systems can be divided into two groups: agent-based and agentless With an

agent-based system a program (agent) is installed on the host server being monitored that is

Trang 20

continually running as a service (on Windows) or a daemon (on Unix-based servers) This agentmonitors the local system and communicates back to a central server Agentless monitoring,

on the other hand, does not require an additional agent to be running on the local server.Monitoring may be done remotely through protocols such as SSH and SNMP, or locally via ascheduled task or cronjob

An agent can often give more visibility of the inner workings of the monitored server However,

an agent requires maintenance of a distributed program, and may use more system resources onthe monitored server than agentless monitoring If there are hundreds of servers, installation andupgrades can be quite time consuming With agentless monitoring, you can have a very quickrollout of monitoring services for even large numbers of servers

Examining Open Source Monitoring

The most popular monitoring systems are open source frameworks Though Nagios is the mostpopular alerting system and Cacti is the most popular graphing system, there are other moni-toring systems that are good enough to deserve mention This section will discuss the followingmonitoring frameworks:

One of the most widely used open source alerting systems is Nagios (pronounced

‘‘NAH-ghee-ose’’) Nagios was originally developed for use under Linux, though it can beused under other variants of Unix, including BSD systems, Solaris and Mac OS X Nagios canmonitor Windows servers, but the centralized Nagios server does not run natively on Windows.Nagios does not do graphing or trending very well The web interface is written in PHP, and thechecks are mostly Perl and shell scripts, though checks can be a script in any language that canrun from the command line

Many operating system distributions include Nagios; if yours does not, it can be downloaded

atwww.nagios.org, where there is also extensive documentation The main features ofNagios are:

■ Graphical web-based user interface

■ Convenient dashboard overview for overall health at-a-glance

■ Supports host, service and network monitoring

636

Trang 21

■ Supports agent-less checks and client-server agents (a daemon running on each clientmachine checked sends information back to the centralized server)

■ Supports agent-less checking of cloud computing environments Supports agent-basedchecking if the agent is installed in the cloud computing environment

■ Text file based configuration (there is a separate nagiosql package that makes the ration web-based)

configu-■ Extensive user community, including many free third-party plugins (www.

nagiosexchange.org)

■ Define custom alerting scripts, with four possible statuses

■ Define custom actions based on status

■ Extremely customizable user permissions

■ Web interface authorization for security

■ Servers can be easily grouped

■ History and basic reporting of alert status

■ Which machines are checked, when they are checked, who is notified and what actionsare taken are all configurable in almost any way possible — you can specify that a machine

is not checked Sundays from 3 pm to 4 pm, or a group of machines should e-mail alertsfrom 6 pm to 8 am and send pages from 8 am to 6 pm

■ One-off downtimes can be scheduled

■ Custom paging escalations

■ Acknowledgement of a problem can be optional or required

■ Dependencies can be set up, reducing pager storms when a parent service goes down

■ Very basic network topology map

■ Screenshots of Nagios are available atwww.nagios.org/about/screenshots.php

■ Virtual appliance for VMWare are available atwww.vmware.com/appliances/

Trang 22

■ Agent-less monitoring

■ Due to agent-less monitoring, implicitly supports checking of cloud computing ments

environ-■ Web-based configuration management

■ Extensive user community, including many free templates (http://forums.cacti net/about15067.html)

■ Define custom data-gathering scripts

■ Define custom graphs

■ User-defined preferences

■ History and extensive reporting of data gathered

■ View many graphs at once

■ View multiple servers on a graph

■ View multiple services on a graph

■ Customizable graph type (line, area, stack, etc) and graph order

■ Easy drag-and-drop to zoom on any graph

■ View/edit raw data

■ Screenshots of cacti are available atwww.cacti.net/screenshots.php

■ Virtual appliances for VMWare, Parallels, Qemu, KVM, VirtualIron and Microsoft Virtual

PC are available athttp://virtualappliances.net/products/cacti.php

There are plugins to Cacti which give basic alerting functionality, but nowhere near the ity of Nagios However, Cacti is the most comprehensive open source graphing tool available.There is also no way to define a custom set of times for when to gather data for graphingwith Cacti

flexibil-Baron Schwartz, MySQL community member and co-author of High Performance MySQL, 2 nd

edition, developed a set of Cacti templates for monitoring MySQL (www.xaprb.com/blog/ tag/cacti-templates/)

Trang 23

available, and there is also a platform-independent version, which uses the Java Runtime ronment (JRE) already installed on your system Hyperic HQ can run SNMP checks, but is notdependent on SNMP

Envi-The main features of Hyperic HQ are:

■ Supports client-server agents (a daemon running on each client machine checked sendsinformation back to the centralized server)

■ Explicitly supports checking of cloud computing environments including Amazon WebServices and Google App Engine

■ Auto-discovery of services on a host (once an agent is deployed)

■ Web-based configuration management

■ Many built-in checks, including hundreds of MySQL checks Seewww.hyperic.com/

products/managed/mysql-management.htmfor the impressive list of metrics onreliability, transactions, connections, SQL commands, configuration changes, resourceutilization, queries, I/O, and tables

■ Define custom alerting scripts

■ Manually performANALYZE TABLE,CHECK TABLE,OPTIMIZE TABLEandREPAIR TABLE

from the Hyperic HQ interface

■ View multiple servers on a graph

■ View multiple services on a graph

■ Dependencies can be set up, reducing pager storms in the event that a parent service goesdown

■ Screenshots available atwww.hyperic.com/demo/gallery.html

■ VMWare virtual appliance versionHyperic does have a user community and user-contributed plugins, though the user community

is not as large as that of Nagios There are not many user-contributed plugins, but that is likely

Trang 24

due to the comprehensive nature of Hyperic’s built-in checks There is an Enterprise edition forpurchase that adds in many features including automated failover, more flexible alerting, auto-mated corrective actions, scheduled maintenance, and role-based access control

Hyperic HQ is an appropriate solution for an organization that requires graphing and alerting

to be a part of the same application Although the alerting functionality in the non-Enterpriseversion is similar to that of Cacti’s capabilities, Hyperic HQ has more out-of-the-box graphs forMySQL

OpenNMS

The open network management system, or OpenNMS, is a distributed graphing and alerting tem Written in Java, XML and XSL, it is available fromwww.opennms.orgfor Linux, Solaris,BSD, Mac OS X, and Windows OpenNMS checks a number of protocols, though it is depen-dent on SNMP for many checks

sys-The main features of OpenNMS are:

■ Auto-discovery of services on a host or IP address range

■ Agent-less monitoring

■ User-defined preferences

■ Zoom by time-period on any graph

■ Active user community

640

Trang 25

■ Dependencies can be set up, reducing pager storms in the event that a parent servicegoes down

■ Extensive network topology map

■ Official demo available fromhttp://demo.opennms.org/opennms/(username:

demo, password: demo)OpenNMS does not seem to have many custom checks, and it is unclear whether that is becauseSNMP can handle most of the checks, or if there is no easy way such as a script, plug-in or tem-plate to share a check

Zenoss Core

Zenoss Core is a Python-based monitoring system that runs on Linux, Solaris, BSDand Mac OS X environments Data can be gathered via SNMP or via the Zenoss Plu-gins, platform-specific python monitoring scripts Zenoss Core can be downloaded at

www.zenoss.com/download/links

The main features of Zenoss Core are:

■ Supports agentless checks and client-server agents (a daemon running on each clientmachine checked sends information back to the centralized server)

■ Supports agent-less checking of cloud computing environments Supports agent-basedchecking if the agent is installed in the cloud computing environment

■ Auto-discovery of services on a host

■ Very active user community, with many user-contributed customizations(www.zenoss.com/community/wiki/tips-and-tricks/)

Tiêu đề	Query analysis and index tuning
Trường học	Unknown University
Chuyên ngành	Database Management
Thể loại	Lecture Notes
Năm xuất bản	Unknown Year
Thành phố	Unknown City

Định dạng
Số trang	50
Dung lượng	1,38 MB