Intel Xeon 3.6 2MB vs AMD Opteron 252 Database Test
by Jason Clark & Ross Whitehead on February 14, 2005 8:00 AM EST- Posted in
- IT Computing
"Order Entry" Stress Test: Measuring Enterprise Class Performance
One complaint that we've historically received regarding our Forums database test was that it isn't strenuous enough for some of the Enterprise customers to make a good decision based on the results.
In our infinite desire to please everyone, we worked very closely with a company that could provide us with a truly Enterprise Class SQL stress application. We cannot reveal the identity of the Corporation that provided us with the application because of non-disclosure agreements in place. As a result, we will not go into specifics of the application, but rather provide an overview of its database interaction so that you can grasp the profile of this application, and understand the results of the tests better (and how they relate to your database environment).
We will use an Order Entry system as an analogy for how this test interacts with the database. All interaction with the database is via stored procedures. The main stored procedures used during the test are:
sp_AddOrder - inserts an Order
sp_AddLineItem - inserts a Line Item for an Order
sp_UpdateOrderShippingStatus - updates a status to "Shipped"
sp_AssignOrderToLoadingDock - inserts a record to indicate from which Loading Dock the Order should be shipped
sp_AddLoadingDock - inserts a new record to define an available Loading Dock
sp_GetOrderAndLineItems - selects all information related to an Order and its Line Items
The above is only intended as an overview of the stored procedure functionality; obviously, the stored procedures perform other validation, and audit operations.
Each Order had a random number of Line Items, ranging from one to three. Also randomized was the Line Items chosen for an order, from a pool of approximately 1500 line items.
Each test was run for 10 minutes and was repeated three times. The average between the three tests was used. The number of Reads to Writes was maintained at 10 reads for every write. We debated for a long while about which ratio of reads to writes would best serve the benchmark, and we decided that there was no correct answer. So, we went with 10.
The application was developed using C#, and all database connectivity was accomplished using ADO.NET and 20 threads - 10 for reading and 10 for inserting.
So, to ensure that IO was not the bottleneck, each test was started with an empty database and expanded to ensure that auto-grow activity did not occur during the test. Additionally, a gigabit switch was used between the client and the server. During the execution of the tests, there were no applications running on the server or monitoring software. Task Manager, Profiler, and Performance Monitor were used when establishing the baseline for the test, but never during execution of the tests.
At the beginning of each platform, both the server and client workstation were rebooted to ensure a clean and consistent environment. The database was always copied to the 8-disk RAID 0 array with no other files present to ensure that file placement and fragmentation was consistent between runs. In between each of the three tests, the database was deleted, and the empty one was copied again to the clean array. SQL Server was not restarted.
Facebook
Twitter
RSS
97 Comments
View All Comments
Zan Lynx - Monday, February 14, 2005 - link
I see Viditor explained what he really wanted. That was my first comment and by the time I'd filled out all the forms and received the email with my password he had already explained. Sorry. Please ignore me.Zan Lynx - Monday, February 14, 2005 - link
Viditor, the test hardware used 8GB RAM for both the Xeon and Opteron systems according to page 2.Viditor - Monday, February 14, 2005 - link
Jason - Let me expand on my request...Because there is still no hardware IOMMU on Xeon chipsets, I believe they must use PAE for 64bit addressing over 4GB, however Opteron doesn't have this problem and can address directly up to 128GB.
I would very much like to see the results of a comparison on the same testbed you used for this article (8GB Ram) to compare and see how much this effects performance as this seems a very typical model to me.
Cheers!
Viditor - Monday, February 14, 2005 - link
Jason - Very well done test and article!I too would be very interested in a 64bit Linux (or even Windows Beta) test with that configuration...
One of the things I am anxious to see is Xeons reaction to >4GB of ram on its performance. There are still NO results (that I have seen) with that configuration.
Cheers, and thanks for the article.
sri2000 - Monday, February 14, 2005 - link
Someone mentioned adding other database functions to provide different kind of stresses. How about using SQL Server Data Transfromation Services (DTS) to perform a variety of mass imports/exports from the test database?You could also perform some Full-Text searches mixed in with the regular queries on appropriately indexed tables - though those are really disk intensive rather than CPU-intensive (though the CPU usage does spike significantly when these queries are run).
I also wonder if adding queries which hit Views in addition to regular tables would affect anything, the result being that you're essentially running nested queries (though this doesn't likely reflect the type of usage seen in your forums, which was the basis of this test).
By the same token, having queries that use wildcards, user functions, sub-queries, etc (rather than just simple selects & inserts) will also add complexity to the searches & might affect the results.
Marlin1975 - Monday, February 14, 2005 - link
Like someone else pointed out, when will you do some test to see what the SSE3 did for AMD.Also what were the temps on both of the NEW Cpus. Haveing hundreds of them in a server room can cost a arm and a leg to keep cool, so I think temps do matter here.
fitten - Monday, February 14, 2005 - link
#39, we don't need to point out that 64-bit Intel P4 Xeons have been out and available for a while even though WindowsXP64 isn't available yet. You can run the RC WindowsXP64 on those and on Opterons/Athlon64s.rgb - Monday, February 14, 2005 - link
I don't think the BIOS of the test machine was adapted to Revision E Opterons.I adapted LinuxBIOS to the Rev E stepping last week and the 1 GHz support is really the easiest thing (was already present in revision D processors). Changing the HT speed while the operating system is running is _very_ difficult. It requires a reset or LDTSTOP on both CPUs for the new frequency to be effective, so this is normally done a boot time in the BIOS. I guess ntune does not really change the HT frequency.
In addition Revision E has a number of errata fixed which result in improved performance (for example Errata 94).
The most important point is the new memory controller mode that reduces the DRAM bank conflicts. It improves STREAM benchmarks scores around 30%. This modes has to be automatically enabled by the BIOS, so please rerun the benchmark on a mainboard that supports Rev E processors.
Quanticles - Monday, February 14, 2005 - link
I'd really want to see tests run on Linux, even if it is 32-bit. There are too many Windows programs that are tailored to Intel processors.I dont need to point out that Microsoft is delaying the 64 bit version of Windows until Intel has their 64 bit processor come out. If they're going to delay like that then I wonder how well the Opteron will preform on it.
Phiro - Monday, February 14, 2005 - link
Jason: Ignore all the 64-bit idiots. Please keep supplying 32-bit sql benchmarks for a LONG time - in the real world 99.5% of production dbs are running on 32-bit sql servers and that number will remain quite high for a long, long time no matter how fast 64-bit takes off.