[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: CRC -> MD5

Howard Chu writes:
> I had considered MD5 before (especially since we already had code for
> it) but it was slower, and we're not looking for cryptographic
> assurances or hash distribution anyway.

Yes, I was thinking mostly of killing unnecessary code and exposed
features.  After a brief test, I could see no good reason to keep CRC:
Neither speed nor quality.  On my machine, anyway.

> Basically all of these crypto hash functions are overkill, in terms of
> hash size and computation. We're only looking to detect casual misuse
> or corruption, not malicious deception.

Yes.  The main reason I see to use either CRC or MD5 is that they are
likely to be installed somewhere so non-programmers can use them.  We
could use a faster hash like MurmurHash3, but would then need to provide
a 'slaputil hash' command or something for use by shell programs.

> I didn't really spend a lot of time comparing the two functions'
> speed. But even with the memory access bottleneck, I would guess that
> on a loaded system with many threads running, the algorithm with fewer
> instructions is the better choice. Have you measured the throughput
> when multiple threads are executing?

On my 32-bit host, MD5 on a threaded test program had 90-95% of CRC's
throughput instead of 105% or whatever it was unthreaded.  OTOH crc32()
from linking -lz gave ~275%.  OTOH I'd expect MD5 to be more costly on
an older or cheaper machine where the which hasn't outpaced memory speed
as much as modern workhorses.