Difference between revisions of "DbConfig"
m |
m |
||
| Line 5: | Line 5: | ||
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9. | We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9. | ||
| − | + | === Prestuff === | |
| − | + | In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat: | |
| + | <pre>cat xcu_file_list | xargs cat | cat > /dev/null</pre> | ||
| − | === | + | === Ramdisk === |
| − | === Leveraged Backend | + | We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache. |
| + | |||
| + | === Const Char* === | ||
| + | We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk. | ||
| + | |||
| + | == Leveraged Backend == | ||
== Futures == | == Futures == | ||
Revision as of 23:36, 5 February 2006
Contents
Motivation
In our initial investigations of cold-start performance, we theorized that the impact of having hundreds of small xml files to be crawled had a negative impact on performance. We reasoned that combining the hundreds of files into one or two containers would greatly improve the ability of the buffer-cache to work effectively and reduce the startup time.
Experiments
We performed several experiments to get an estimate of the performance gains that could be had. We tried three methodologies that all indicated that we could expect roughly a 1 second improvement on a cold-start of writer. Our test machine for these experiments was a 3.2Ghz Pentium 4HT with 1 gigabyte of RAM, a 120GB 7200rpm SATA drive, running NLD9.
Prestuff
In this experiment, we pre-stuffed the buffer-cache with the xcu files before startup. This was accomplished by running cat:
cat xcu_file_list | xargs cat | cat > /dev/null
Ramdisk
We created two ramdisks and put the configuration data in them. This was a little coarser, because the contents registry was the ramdisk, which also includes the cache.
Const Char*
We created a header file that had all of the contents of the xcu files as const character strings. A simple search function was crafted that took a URL and found the appropriate string. We changed the implimentation of XLayer.readData to look into the strings instead of the disk. Writes were left going to the disk.
