What kind of compression algorithm is this?

2018-06-03 06:40:50

So I have a couple of compressed files, and the uncompressed versions as well. I do not have the software that created these files originally. I'm trying to figure out what the underlying algorithm is -- can you figure it out? Initially, I thought it might be some LZW variant, but I'm not sure. The data seems to make more sense when broken up into 6-bit words -- I see lots of repeating patterns then.

The two files are very similar, and the uncompressed versions differ by only a few bytes -- that could help establish where those differing bytes are in the compressed files. I've highlighted the differences.

Compressed file #1:

02 02 01 17 0E 11 92 14 C0 55 52 44 FF BC AE 47 DB E1 05 42 F8 70 DE 57 23 FF
54 1A 55 3D BF 54 10 E3 38 0C B2 FB C4 92 1C 20 DE 57 23 FF 54 1A 55 3D BE 5E
4C 96 B2 0E 32 80 CB 2F BC 48 70 83 79 5C 8F FD 50 69 54 F6 F9 96 48 A9 07 19
C2 30 F0 E1 BC AE 47 FE A8 34 AA 7B 7E 32 BF E5 1F EE A8 48 CA 11 87 87 0D E5
72 3F F5 41 A5 53 DB E5 24 5D F8 CA FF 4C B1 13 8C 71 18 7B C3 86 F2 B9 1F FA
A0 D2 A9 ED FD 55 97 BA 22 32 C0 CB 2F BC

Compressed file #2:

02 02 01 17 0E 11 92 14 C0 55 52 44 FF BC AE 47 DB E1 05 42 F8 70 DE 57 23 FF
54 1A 55 3D BF 54 10 E3 38 0D 36 D4 04 92 1C 20 DE 57 23 FF 54 1A 55 3D BE 5E
4C 96 B2 0E 32 80 D3 6D 40 48 70 83 79 5C 8F FD 50 69 54 F6 F9 96 48 A9 07 19
C2 30 F0 E1 BC AE 47 FE A8 34 AA 7B 7E 32 BF E5 1F EE A8 48 CA 11 87 87 0D E5
72 3F F5 41 A5 53 DB E5 24 5D F8 CA FF 4C B1 13 8C 71 18 7B C3 86 F2 B9 1F FA
A0 D2 A9 ED FD 55 97 BA 22 32 C0 D3 6D 40

Uncompressed file #1:

20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 2A 2A 2A 2A 2A 20 47 52 41 4E 44 20 54 4F 54 41 4C 53 20
2A 2A 2A 2A 2A 0D 0A 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 54 4F 54 41 4C 20 52 45 43 4F 52 44
53 20 52 45 41 44 20 20 20 20 20 20 20 20 20 20 20 20 20 20 32 32 38 37 0D 0A
0D 0A 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 54 4F 54 41 4C 20 52 45 43 4F 52 44 53 20 42 59 50
41 53 53 45 44 20 20 20 20 20 20 20 20 20 20 32 32 38 37 0D 0A 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 54 4F 54 41 4C 20 52 45 43 4F 52 44 53 20 43 48 41 4E 47 45 44 20 20 20
20 20 20 20 20 20 20 20 20 20 20 30 0D 0A 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 54 4F 54 41 4C
20 52 45 43 4F 52 44 53 20 4E 4F 54 20 4F 4E 20 58 52 45 46 20 20 20 20 20 20
20 20 20 20 30 0D 0A 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 54 4F 54 41 4C 20 52 45 43 4F 52 44
53 20 42 41 4E 4B 20 4E 4F 54 20 46 4F 55 4E 44 20 20 20 20 20 20 20 30 0D 0A
0D 0A 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 54 4F 54 41 4C 20 52 45 43 4F 52 44 53 20 57 52 49
54 54 45 4E 20 20 20 20 20 20 20 20 20 20 20 32 32 38 37

Uncompressed file #2:

20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 2A 2A 2A 2A 2A 20 47 52 41 4E 44 20 54 4F 54 41 4C 53 20
2A 2A 2A 2A 2A 0D 0A 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 54 4F 54 41 4C 20 52 45 43 4F 52 44
53 20 52 45 41 44 20 20 20 20 20 20 20 20 20 20 20 20 20 20 33 34 33 39 0D 0A
0D 0A 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 54 4F 54 41 4C 20 52 45 43 4F 52 44 53 20 42 59 50
41 53 53 45 44 20 20 20 20 20 20 20 20 20 20 33 34 33 39 0D 0A 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 54 4F 54 41 4C 20 52 45 43 4F 52 44 53 20 43 48 41 4E 47 45 44 20 20 20
20 20 20 20 20 20 20 20 20 20 20 30 0D 0A 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 54 4F 54 41 4C
20 52 45 43 4F 52 44 53 20 4E 4F 54 20 4F 4E 20 58 52 45 46 20 20 20 20 20 20
20 20 20 20 30 0D 0A 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 20 20 20 20 20 54 4F 54 41 4C 20 52 45 43 4F 52 44
53 20 42 41 4E 4B 20 4E 4F 54 20 46 4F 55 4E 44 20 20 20 20 20 20 20 30 0D 0A
0D 0A 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
20 20 20 20 20 20 20 20 20 54 4F 54 41 4C 20 52 45 43 4F 52 44 53 20 57 52 49
54 54 45 4E 20 20 20 20 20 20 20 20 20 20 20 33 34 33 39

As you can see, the output files are just plain ASCII text files. Any ideas?

This seems to be some proprietary encoding format, designed to shave some bits of specific types of messages.

It operates on 8 bit (ascii) input and outputs a bit stream using a mixture of a 5 and 6 bit token set, including some control characters.

The following tokens can be identified:

// 5 bit tokens:
00000 switch to 6 bit mode
00011 take the following 6 bits as N, and output N spaces
00100 A
00101 B
.....
11101 Z
11110 <crlf>
11111 space

// 6 bit tokens:
000001 switch to 5 bit mode
000011 take the following 6 bits as N, and output N spaces
001001 <crlf>
011000 1
011001 2
......
100000 9

// pure speculation:
010111 0
010010 *
000110 repeat the next 6 bit char N times
001100 space
00001 skip 3 bits, take the next 8 bits as ascii, and output N times

Without more examples it is hard to determine what happens at the beginning of the stream. It might be some magic value, or could contain some control values.

链接地址: http://www.djcxy.com/p/11260.html

上一篇: 如何AngularJS日期从json

下一篇: 这是什么样的压缩算法？