1LZMA compression 2---------------- 3Version: 9.35 4 5This file describes LZMA encoding and decoding functions written in C language. 6 7LZMA is an improved version of famous LZ77 compression algorithm. 8It was improved in way of maximum increasing of compression ratio, 9keeping high decompression speed and low memory requirements for 10decompressing. 11 12Note: you can read also LZMA Specification (lzma-specification.txt from LZMA SDK) 13 14Also you can look source code for LZMA encoding and decoding: 15 C/Util/Lzma/LzmaUtil.c 16 17 18LZMA compressed file format 19--------------------------- 20Offset Size Description 21 0 1 Special LZMA properties (lc,lp, pb in encoded form) 22 1 4 Dictionary size (little endian) 23 5 8 Uncompressed size (little endian). -1 means unknown size 24 13 Compressed data 25 26 27 28ANSI-C LZMA Decoder 29~~~~~~~~~~~~~~~~~~~ 30 31Please note that interfaces for ANSI-C code were changed in LZMA SDK 4.58. 32If you want to use old interfaces you can download previous version of LZMA SDK 33from sourceforge.net site. 34 35To use ANSI-C LZMA Decoder you need the following files: 361) LzmaDec.h + LzmaDec.c + Types.h 37 38Look example code: 39 C/Util/Lzma/LzmaUtil.c 40 41 42Memory requirements for LZMA decoding 43------------------------------------- 44 45Stack usage of LZMA decoding function for local variables is not 46larger than 200-400 bytes. 47 48LZMA Decoder uses dictionary buffer and internal state structure. 49Internal state structure consumes 50 state_size = (4 + (1.5 << (lc + lp))) KB 51by default (lc=3, lp=0), state_size = 16 KB. 52 53 54How To decompress data 55---------------------- 56 57LZMA Decoder (ANSI-C version) now supports 2 interfaces: 581) Single-call Decompressing 592) Multi-call State Decompressing (zlib-like interface) 60 61You must use external allocator: 62Example: 63void *SzAlloc(void *p, size_t size) { p = p; return malloc(size); } 64void SzFree(void *p, void *address) { p = p; free(address); } 65ISzAlloc alloc = { SzAlloc, SzFree }; 66 67You can use p = p; operator to disable compiler warnings. 68 69 70Single-call Decompressing 71------------------------- 72When to use: RAM->RAM decompressing 73Compile files: LzmaDec.h + LzmaDec.c + Types.h 74Compile defines: no defines 75Memory Requirements: 76 - Input buffer: compressed size 77 - Output buffer: uncompressed size 78 - LZMA Internal Structures: state_size (16 KB for default settings) 79 80Interface: 81 int LzmaDecode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen, 82 const Byte *propData, unsigned propSize, ELzmaFinishMode finishMode, 83 ELzmaStatus *status, ISzAlloc *alloc); 84 In: 85 dest - output data 86 destLen - output data size 87 src - input data 88 srcLen - input data size 89 propData - LZMA properties (5 bytes) 90 propSize - size of propData buffer (5 bytes) 91 finishMode - It has meaning only if the decoding reaches output limit (*destLen). 92 LZMA_FINISH_ANY - Decode just destLen bytes. 93 LZMA_FINISH_END - Stream must be finished after (*destLen). 94 You can use LZMA_FINISH_END, when you know that 95 current output buffer covers last bytes of stream. 96 alloc - Memory allocator. 97 98 Out: 99 destLen - processed output size 100 srcLen - processed input size 101 102 Output: 103 SZ_OK 104 status: 105 LZMA_STATUS_FINISHED_WITH_MARK 106 LZMA_STATUS_NOT_FINISHED 107 LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK 108 SZ_ERROR_DATA - Data error 109 SZ_ERROR_MEM - Memory allocation error 110 SZ_ERROR_UNSUPPORTED - Unsupported properties 111 SZ_ERROR_INPUT_EOF - It needs more bytes in input buffer (src). 112 113 If LZMA decoder sees end_marker before reaching output limit, it returns OK result, 114 and output value of destLen will be less than output buffer size limit. 115 116 You can use multiple checks to test data integrity after full decompression: 117 1) Check Result and "status" variable. 118 2) Check that output(destLen) = uncompressedSize, if you know real uncompressedSize. 119 3) Check that output(srcLen) = compressedSize, if you know real compressedSize. 120 You must use correct finish mode in that case. */ 121 122 123Multi-call State Decompressing (zlib-like interface) 124---------------------------------------------------- 125 126When to use: file->file decompressing 127Compile files: LzmaDec.h + LzmaDec.c + Types.h 128 129Memory Requirements: 130 - Buffer for input stream: any size (for example, 16 KB) 131 - Buffer for output stream: any size (for example, 16 KB) 132 - LZMA Internal Structures: state_size (16 KB for default settings) 133 - LZMA dictionary (dictionary size is encoded in LZMA properties header) 134 1351) read LZMA properties (5 bytes) and uncompressed size (8 bytes, little-endian) to header: 136 unsigned char header[LZMA_PROPS_SIZE + 8]; 137 ReadFile(inFile, header, sizeof(header) 138 1392) Allocate CLzmaDec structures (state + dictionary) using LZMA properties 140 141 CLzmaDec state; 142 LzmaDec_Constr(&state); 143 res = LzmaDec_Allocate(&state, header, LZMA_PROPS_SIZE, &g_Alloc); 144 if (res != SZ_OK) 145 return res; 146 1473) Init LzmaDec structure before any new LZMA stream. And call LzmaDec_DecodeToBuf in loop 148 149 LzmaDec_Init(&state); 150 for (;;) 151 { 152 ... 153 int res = LzmaDec_DecodeToBuf(CLzmaDec *p, Byte *dest, SizeT *destLen, 154 const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode); 155 ... 156 } 157 158 1594) Free all allocated structures 160 LzmaDec_Free(&state, &g_Alloc); 161 162Look example code: 163 C/Util/Lzma/LzmaUtil.c 164 165 166How To compress data 167-------------------- 168 169Compile files: 170 Types.h 171 Threads.h 172 LzmaEnc.h 173 LzmaEnc.c 174 LzFind.h 175 LzFind.c 176 LzFindMt.h 177 LzFindMt.c 178 LzHash.h 179 180Memory Requirements: 181 - (dictSize * 11.5 + 6 MB) + state_size 182 183Lzma Encoder can use two memory allocators: 1841) alloc - for small arrays. 1852) allocBig - for big arrays. 186 187For example, you can use Large RAM Pages (2 MB) in allocBig allocator for 188better compression speed. Note that Windows has bad implementation for 189Large RAM Pages. 190It's OK to use same allocator for alloc and allocBig. 191 192 193Single-call Compression with callbacks 194-------------------------------------- 195 196Look example code: 197 C/Util/Lzma/LzmaUtil.c 198 199When to use: file->file compressing 200 2011) you must implement callback structures for interfaces: 202ISeqInStream 203ISeqOutStream 204ICompressProgress 205ISzAlloc 206 207static void *SzAlloc(void *p, size_t size) { p = p; return MyAlloc(size); } 208static void SzFree(void *p, void *address) { p = p; MyFree(address); } 209static ISzAlloc g_Alloc = { SzAlloc, SzFree }; 210 211 CFileSeqInStream inStream; 212 CFileSeqOutStream outStream; 213 214 inStream.funcTable.Read = MyRead; 215 inStream.file = inFile; 216 outStream.funcTable.Write = MyWrite; 217 outStream.file = outFile; 218 219 2202) Create CLzmaEncHandle object; 221 222 CLzmaEncHandle enc; 223 224 enc = LzmaEnc_Create(&g_Alloc); 225 if (enc == 0) 226 return SZ_ERROR_MEM; 227 228 2293) initialize CLzmaEncProps properties; 230 231 LzmaEncProps_Init(&props); 232 233 Then you can change some properties in that structure. 234 2354) Send LZMA properties to LZMA Encoder 236 237 res = LzmaEnc_SetProps(enc, &props); 238 2395) Write encoded properties to header 240 241 Byte header[LZMA_PROPS_SIZE + 8]; 242 size_t headerSize = LZMA_PROPS_SIZE; 243 UInt64 fileSize; 244 int i; 245 246 res = LzmaEnc_WriteProperties(enc, header, &headerSize); 247 fileSize = MyGetFileLength(inFile); 248 for (i = 0; i < 8; i++) 249 header[headerSize++] = (Byte)(fileSize >> (8 * i)); 250 MyWriteFileAndCheck(outFile, header, headerSize) 251 2526) Call encoding function: 253 res = LzmaEnc_Encode(enc, &outStream.funcTable, &inStream.funcTable, 254 NULL, &g_Alloc, &g_Alloc); 255 2567) Destroy LZMA Encoder Object 257 LzmaEnc_Destroy(enc, &g_Alloc, &g_Alloc); 258 259 260If callback function return some error code, LzmaEnc_Encode also returns that code 261or it can return the code like SZ_ERROR_READ, SZ_ERROR_WRITE or SZ_ERROR_PROGRESS. 262 263 264Single-call RAM->RAM Compression 265-------------------------------- 266 267Single-call RAM->RAM Compression is similar to Compression with callbacks, 268but you provide pointers to buffers instead of pointers to stream callbacks: 269 270SRes LzmaEncode(Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen, 271 const CLzmaEncProps *props, Byte *propsEncoded, SizeT *propsSize, int writeEndMark, 272 ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig); 273 274Return code: 275 SZ_OK - OK 276 SZ_ERROR_MEM - Memory allocation error 277 SZ_ERROR_PARAM - Incorrect paramater 278 SZ_ERROR_OUTPUT_EOF - output buffer overflow 279 SZ_ERROR_THREAD - errors in multithreading functions (only for Mt version) 280 281 282 283Defines 284------- 285 286_LZMA_SIZE_OPT - Enable some optimizations in LZMA Decoder to get smaller executable code. 287 288_LZMA_PROB32 - It can increase the speed on some 32-bit CPUs, but memory usage for 289 some structures will be doubled in that case. 290 291_LZMA_UINT32_IS_ULONG - Define it if int is 16-bit on your compiler and long is 32-bit. 292 293_LZMA_NO_SYSTEM_SIZE_T - Define it if you don't want to use size_t type. 294 295 296_7ZIP_PPMD_SUPPPORT - Define it if you don't want to support PPMD method in AMSI-C .7z decoder. 297 298 299C++ LZMA Encoder/Decoder 300~~~~~~~~~~~~~~~~~~~~~~~~ 301C++ LZMA code use COM-like interfaces. So if you want to use it, 302you can study basics of COM/OLE. 303C++ LZMA code is just wrapper over ANSI-C code. 304 305 306C++ Notes 307~~~~~~~~~~~~~~~~~~~~~~~~ 308If you use some C++ code folders in 7-Zip (for example, C++ code for .7z handling), 309you must check that you correctly work with "new" operator. 3107-Zip can be compiled with MSVC 6.0 that doesn't throw "exception" from "new" operator. 311So 7-Zip uses "CPP\Common\NewHandler.cpp" that redefines "new" operator: 312operator new(size_t size) 313{ 314 void *p = ::malloc(size); 315 if (p == 0) 316 throw CNewException(); 317 return p; 318} 319If you use MSCV that throws exception for "new" operator, you can compile without 320"NewHandler.cpp". So standard exception will be used. Actually some code of 3217-Zip catches any exception in internal code and converts it to HRESULT code. 322So you don't need to catch CNewException, if you call COM interfaces of 7-Zip. 323 324--- 325 326http://www.7-zip.org 327http://www.7-zip.org/sdk.html 328http://www.7-zip.org/support.html 329