GCFScape v1.6.0 And HLLib v2.0.2 - NemPosted: Nov 10th, 2006 - 5:41:14 pm

I mentioned previously that my attempt to add a fragmentation property to HLLib revealed a surprising amount of GCF fragmentation, even after Steam's built in GCF defragmenter had been run. As a result I've released a new version of GCFScape (complete change-log) and HLLib with my own implementation of a GCF defragmenter. As always you can find GCFScape here.

The results of HLLib's defragmenter were better than I had expected with an average speedup on HLLib tests of 2.62 and an average speedup on Steam tests of 2.32 (for a total average speedup of 2.47). The baseline tests were all conducted on Day of Defeat: Source GCF files that had been run through Steam's defragmenter (which claimed 0% fragmentation even though I calculated an average of 8.38%). I restarted my computer between tests to insure that it was in a similar state for each test. The complete results are as follows:

Test Results
Test Library Fragmented Time Defragmented Time Speedup
Load dod_jagd HLLib (Crafty) 1:07 0:58 1.16
Load dod_anzio Steam 1:52 1:26 1.30
Validate DOD: Source HLLib (GCFScape) 8:01 1:58 4.08
Validate DOD: Source Steam 6:29 1:56 3.35

Of particular interest is the 3.35 speedup for DOD: Source validation through Steam. This is because validation is more IO bound than CPU bound meaning the speedup is indicative of Steam's raw file IO speedup. The result of this IO speedup can be seen in the dod_anzio test where I was able to load dod_anzio 26 seconds faster than before defragmentation. Even the casual gamer should appreciate that.

To defragment your GCF files using GCFScape, do the following:

  1. Shutdown Steam.
  2. Launch GCFScape.
  3. In the Options menu, enable Write Acces and disable Volatile Access.
  4. Open your GCF file.
  5. Select Defragment from the Tools menu.

HLLib uses a crude and slow defragmentation algorithm which moves each file (in the order it appears in the directory) to the start of the GCF file sequentially regardless of whether or not the file is fragmented. This algorithm may be crude, but it insures that each file is 100% defragmented and has the added benefit of arranging files lexicographically such that items such as models, which are made up of multiple files, are placed side by side and can be read with little seeking. The algorithm can be canceled at any time.


1. SlackillerPosted: Nov 12th, 2006 - 12:42:03 am
apple Good Stuff

2. adam0509Posted: Nov 22nd, 2006 - 1:47:37 pm
So, If I understand, It means it move file INSIDE the GCF, but it don't move the entire GCF file ? (and so the GCF file is still fragmented view from the disk...)

3. NemPosted: Nov 22nd, 2006 - 3:37:02 pm
Yes, GCFScape defragments the files contained within a .gcf file, not the .gcf file itself (in your Windows NTFS or FAT filesystem). For the greatest performance gain you should defragment your hard drive with Windows and then your .gcf files with GCFScape.

4. JohnnyFlashPosted: Nov 22nd, 2006 - 6:34:47 pm
Do you think it would be possible for you to diplay the file properties in columns on the right-hand panel?

It would really help out us steam developers :)

5. adam0509Posted: Nov 23rd, 2006 - 3:52:56 pm
Okay thanks for the answer Nem !

(Yep I know, Windows' NTFS & FAT suxx, long life to linux and EXT3 filesystem ;) )

6. NemPosted: Nov 23rd, 2006 - 5:41:47 pm
Do you think it would be possible for you to diplay the file properties in columns on the right-hand panel?

I'll see what I can do (perhaps something like how Windows does it), however, I'm quite swamped at the moment so it may be a week or two.

7. Ay2daLoPosted: Nov 23rd, 2006 - 9:05:40 pm
cooooooool cool

8. PandaKingPosted: Jan 31st, 2007 - 5:04:48 am
Can I download it??myhead

You must be logged in to post a comment.
New users can register here.
Nem's Tools v2.0 © 2006 Ryan Gregg.
Execution time: 0.028824s; Queries: 17.
dishes served.
Powered by The Wavelength.

Valid XHTML 1.0 Transitional Valid CSS