[Date Prev][Date Next]
Re: CRC -> MD5
Howard Chu writes:
> I had considered MD5 before (especially since we already had code for
> it) but it was slower, and we're not looking for cryptographic
> assurances or hash distribution anyway.
Yes, I was thinking mostly of killing unnecessary code and exposed
features. After a brief test, I could see no good reason to keep CRC:
Neither speed nor quality. On my machine, anyway.
> Basically all of these crypto hash functions are overkill, in terms of
> hash size and computation. We're only looking to detect casual misuse
> or corruption, not malicious deception.
Yes. The main reason I see to use either CRC or MD5 is that they are
likely to be installed somewhere so non-programmers can use them. We
could use a faster hash like MurmurHash3, but would then need to provide
a 'slaputil hash' command or something for use by shell programs.
> I didn't really spend a lot of time comparing the two functions'
> speed. But even with the memory access bottleneck, I would guess that
> on a loaded system with many threads running, the algorithm with fewer
> instructions is the better choice. Have you measured the throughput
> when multiple threads are executing?
On my 32-bit host, MD5 on a threaded test program had 90-95% of CRC's
throughput instead of 105% or whatever it was unthreaded. OTOH crc32()
from linking -lz gave ~275%. OTOH I'd expect MD5 to be more costly on
an older or cheaper machine where the which hasn't outpaced memory speed
as much as modern workhorses.