Stress testing is an important element of developing any new database application. There are a number of tools and services available to do stress testing, but the well known, full-featured ones are relatively expensive. There are testing companies who will stress-test your application for you, but they are also expensive.
Many small development teams do not have the budget for these tools or services and have to rely on less reliable, ad-hoc methods to test their applications. Consequently, they risk serious performance problems that might show up after the application is released.
If your team doesn’t have a budget for expensive testing tools, there are still a couple of no-cost tools that can help you do a more thorough job of stressing your database application than having a half dozen developers hammer away at their keyboards for an hour or so.
OSTRESS and SQL Profiler
There are a lot of stress tools that have more capabilities than OSTRESS, but none come at a better price. OSTRESS is a free download from Microsoft and comes in a suite of other useful tools. It lacks some of the bells and whistles of the third-party products, but it will allow you to put a serious load on your database server quite easily.
SQL Profiler is not quite free, but if you are developing for SQL Server it is pretty certain that you have a SQL Server license for some paid edition. If so, you already own a copy of SQL Profiler.
Together, these two tools can help you put a realistic load on your database and catch many problems that might not otherwise been caught before release.
Touching Just the High Points
In this short article, we can’t cover all the details involved in using these tools. Our primary goal is to make you aware of their capabilities for testing your applications in development. If you have an immediate interest in using these tools, you can find detailed instructions easily.
OSTRESS comes in a package with very good documentation. There is also a great deal of documentation on the web concerning the REPLAY options in SQL Profiler. We won’t clutter up this article by repeating all the details that are available elsewhere.
OSTRESS is a command line tool much like SQLCMD, iSQL and the other SQL command line utilities. OSTRESS also shares the same basic set of options that are used by those tools. If you are familiar with any of these SQL command line tools, you should have no difficulty learning the additional options that are specific to OSTRESS.
OSTRESS and the other tools in the download have a number of capabilities we don’t have space to cover. For example, OSTRESS also has a replay facility like SQL Profiler. You may want to look into that, but it is not covered here.
In my opinion, the core benefit of OSTRESS is that it allows you to connect to an instance of SQL Server and execute any number of SQL scripts a configurable number of times by a configurable number of user threads.
For example, this command line will execute every sql script in the C:\temp\ directory 1000 times on 100 different threads. It also outputs all result sets to the directory specified by the ‘o parameter:
- Ostress.exe ‘ic:\temp\*.sql ‘ n100 ‘ r1000 ‘oc:\temp\output
As you can see, this is powerful stuff.
By careful attention to the contents of your scripts, you can test a wide range of functionality. You can pull code out of sql traces for your scripts if you like, or you can write a suite of scripts that will test specific functionality in your application. OSTRESS will supply the horsepower to execute them under conditions of high concurrency and high load.
It is good practice to escalate the number of SQL files, repetitions and threads carefully. I got a bit crazy while testing for this article and brought down my own testing server.
There are many other bells and whistles for tweaking and controlling OSTRESS but it is all in the documentation. There is little point in reproducing that material here because you will need to read the documentation before you use OSTRESS anyway.
You can download Ostress in the RML Utilities package available here. There are 32bit and 64bit versions. Be sure to download the right one for your server.
Profiler is most often used to monitor SQL Servers to identify performance bottlenecks, deadlocks, etc. However, it also has the ability to record activity on a SQL Server and then replay that activity either on the same SQL instance or another instance containing a copy of the database.
This is especially useful when you are planning to upgrade your SQL Server to a new edition on a new hardware configuration. Replaying actual activity from the production server on the new server can uncover application incompatibilities and potential performance problems before you cut over to the new platform.
For example, you can capture a period of high activity on your current production server and replay it on the development server to simulate a production load. This can test both compatibility and the ability of the new platform to handle a typical heavy load on the production server.
Because a lot of data is likely to be changed during the trace you take of the production system, your replay must begin with a copy of the database as it was at the beginning of the trace. This means that you must take a backup just before starting the trace. You can then restore the backup on the test server and replay the trace.
There are a number of issues that complicate matters when you are replaying the trace on a different server:
- No matter how hard you work to make the servers identical, they will not be identical, and that can cause replay problems.
- The database id of the target database is not likely to be the same on the replay server.
- The SQL logins are not likely to be linked to the user accounts in the database.
- The users may not have the same default database on the replay server as they have on the production server.
All of these conditions can cause replay errors.
Microsoft suggests restoring a copy of the production Master database to the replay instance to sync up the login and user items. I personally think that is insane.
If you are upgrading you won’t be able to restore a SQL 2000 master database to SQL 2008, for example. Even if you are testing on the same versions, I would do this only if you are prepared to uninstall SQL Server after the testing and then re-install it before transitioning the test server to be the production server.
There are simpler and less dangerous workarounds for these problems.
- You can sync up logins on the new server using sp_change_users_login ‘Auto_Fix’, ‘userName’, NULL, ‘password’; Look up this function for details of usage.
- The user’s default databases may all be the Master database if their accounts have been created using default options. If not, you can change them on the replay server.
- The database id column in the trace can be filtered out of the replay.
Still, this is a significant amount of work to prepare for a replay. But if you are going to eventually use the replay server as your production database server, you are going to have to map the logins to the user accounts and perhaps map the users to other default databases anyway.
These are common problems. On the web you will find many examples of using these workarounds. However, you might also consider whether the ability to replay a trace is worth the work that it takes. The ability to hammer your server using OSTRESS is relatively simple. You may find that it will do everything you really need to do.
OSTRESS and SQL Profiler are probably not the best tools for stress testing your applications and databases, but they are far better than nothing and they are essentially free.
It is true that SQL Profiler replays require a lot of preparation work if you are replaying on another server, but ostress only requires that you create some relevant sql scripts that will execute in your sql environment.