In order to reduce streaming load off disk we're experimenting with an compression algorithm where we reduce each asset to a 32 bit integer length value and either a 32 bit CRC or a 128 bit MD5 value. The values are then used to allocate an appropriately sized buffer and a search is conducted to produce a final dataset that matches the checksum. With an average asset size of 200KB the average compression ratio for disk assets is approximately 100,000:1 for the CRC based algorithm and 40,000:1 for the MD5 based algorithm.
By performing PS2 DVD reads aligned to 16 sector boundaries we can achieve a minimum raw data throroughput in excess of 2Mb/s on the inner edge of the disk and a maximum thoroughput of 4Mb/s toward the outer edge with the average being around 3Mb/s in the middle. This provides an effective average read rate of 300Gb/s using the CRC based algorithm.
Posted by matt at April 1, 2004 11:57 AMAlien "Carbon Rod Measure?"
Posted by: Azziki at April 2, 2004 08:45 PM