123
-=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- (c) WidthPadding Industries 1987 0|600|0 -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=-
Socoder -> On Topic -> Binary file compression

Fri, 09 Jul 2010, 06:00
HoboBen
I know that some compression algorithms are better suited to specific areas, e.g. text compression, than others.

If you wanted to compress a fairly uneven binary file, perhaps an .exe or a game save, what algorithm/program would you use for best effect? What settings would you use (e.g. ideal dictionary size?).

Currently gzip (GNU Zip) is giving me a 50% filesize reduction, but I'm sure there's a way to get down even further.

-=-=-
blog | work | code | more code
Fri, 09 Jul 2010, 06:39
Jayenkai
Really random stuff is insanely hard to compress.

If you're making a level editor, you can fit the data to suit the compression better, but if it's something you have no control over, it's probably not worth the hassle of trying to find something "Amazing!" to do it.

gzip, lzw, whatever. If you can use them, then do.

They've been building over years, and they're where they are because they're good.

-=-=-
''Load, Next List!''
Fri, 09 Jul 2010, 06:48
HoboBen
Thanks Jay,

The data isn't "completely" random, so I think there's probably a good algorithm out there somewhere that would be optimal for this type of data. I've just tried 7zip and it's got filesizes down to a third.

But yes, you're right, I could rearrange the data to fit the compression better. It's not ideal, because it obviously makes it much harder to read the data back out (the simple data format is a key feature!) but there might be some simpler ways to improve the stream's compression potential which I'll definitely try for. Cheers!

-=-=-
blog | work | code | more code
Fri, 09 Jul 2010, 07:32
JL235
Maximum Compression is a site that ranks different compression software for compressing different types of files.
Fri, 09 Jul 2010, 17:22
HoboBen
Thanks for the link - exactly what I'm looking for. It lead me here, where there are dozens of different types to experiment with. I settled with sr3a. It's very slightly less good than 7zip, but very fast.

-=-=-
blog | work | code | more code