|
|
Search_Blog
|
 |
|
|
|
|
|
Blog_Archive
|
 |
|
|
|
 |
|
|
MS SQL - BIG DBs
|
 |
|
Location: Blogs Developments |
 |
| Posted by: TM Central |
8/28/2007 1:43 PM |
Ok, lets face it - most databases in the world are of a size that who cares how they are designed or how a backup strategy is implemented as long as they have ACID, referential integrity, and some basic data-typing for basic validation of entries, and a routine complete backup (yes, I know, lots of other reasons, but stay tuned, because this is not a theoretical disucssion on databases). With a 2 MB (or even a 20 GB) database, most structure changes can take effect almost immediately and do not tax any harddrive that has been manufatured in the past 4+ years. But what about the 250 GB database?!?! Its huge, it fills most disks (though again this is changing), and changes take hours/days to propogate. Here's some tricks we learned for dealing with the monster DB's (please note that the links may expire, but the practices and "buzzwords" should be around to be googled for a good long time):
- Disk space, disk space, disk space. The more you have, the more changes you can make. Take an index for example. When recreating/changing an index, MSSQL in effect copies the table index. You must have space to do this, the more so when the table is 60+ GB!
- When importing and exporting data out of any DB, look for a bulk-copy or bulk-insert utility with the db server. For our example we'll use MS SQL and its utility: bcp. Bcp can write to a text file by factors-faster than a Select/Text ODBC connection. Bcp (if in SQL 7 when used with Bulk Copy Option and Tablock regardless of version), will be even faster by not logging the Transactions to the Log File. This of course means you should either have a Simple Recovery plan (i.e., no log baks - all or nothing) or fully back the DB up immediately after to ensure data recovery. See http://msdn2.microsoft.com/en-us/library/ms190203.aspx for more info on Full/Simple Recoveries.
- Using a bulk-copy product, many will have an API or CLI for use in programming and/or command line. This can automate moves of large data off the server via text or backup db. When using an API with one of these for many of them, you can wrap multiple transactions in a single connection transaction (see http://msdn2.microsoft.com/en-us/library/tchktcdk(VS.80).aspx for bcp/.NET transactions).
- Lots of helpful information about figuring out db, log, and table space abounds on the web, especially if you know where to look and what terms to use:
- USE {database}
- go
- DBCC SQLPERF (LOGSPACE) - get log space definitions
- EXEC sp_spaceused [table],[@updateusage='TRUE'] - get table/db space usage.
- exec sp_helpdb [logical databasename] | [filename] - more information on db or file space and general info.
- DBCC LOGINFO([database name]) - detailed information on the log pages that make up the log including their status.
- DBCC REINDEX [table],[indexname],[fillfactor] - reindexes the table or db specified. Keep in mind that a reindex essestially drops and rebuilds the index and thus does a table-lock on the affected table/db. (See (http://msdn2.microsoft.com/en-US/library/aa258828(SQL.80).aspx for more info). You use DBCC REINDEXDEFRAG, but it does not do a full rebuild (thus also allowing user action in table). See http://www.sql-server-performance.com/tips/dbcc_commands_p1.aspx for more information.
|
|
| Permalink |
Trackback |
|
|
 |
|