Mark Adler [Wed, 30 Mar 2022 18:14:53 +0000 (11:14 -0700)]
Correct incorrect inputs provided to the CRC functions.
The previous releases of zlib were not sensitive to incorrect CRC
inputs with bits set above the low 32. This commit restores that
behavior, so that applications with such bugs will continue to
operate as before.
Mark Adler [Sat, 1 Jan 2022 20:09:30 +0000 (12:09 -0800)]
Fix unztell64() in minizip to work past 4GB. (Daniël Hörchner)
The issue is that unztell64() does not return the correct value if
the position in the current file (in the ZIP archive) is beyond 4
GB. The cause is that unzReadCurrentFile() does not account for
pfile_in_zip_read_info->stream.total_out at line 1854 of unzip.c
wrapping around (it is a 32-bit variable). So, on line 1860
uTotalOutAfter can be *less* than uTotalOutBefore, propagating the
wraparound to uOutThis, which in turn is added to
pfile_in_zip_read_info->total_out_64. That has the effect of
subtracting 4 GB.
Mark Adler [Mon, 18 Feb 2019 04:45:53 +0000 (20:45 -0800)]
Use ARM crc32 instructions if the ARM architecture has them.
The ARM crc32 instructions will be used only if an architecture is
explicitly specified at compile time that has those instructions.
For example, -march=armv8.1-a or -march=armv8-a+crc, or if the
machine being compiled on has the instructions, -march=native.
Mark Adler [Mon, 18 Feb 2019 03:48:57 +0000 (19:48 -0800)]
Add use of the ARMv8 crc32 instructions when requested.
Define the macro Z_ARM_CRC32 at compile time to use the ARMv8
(aarch64) crc32x and crc32b instructions. This code does not check
for the presence of the crc32 instructions. Those instructions are
optional for ARMv8.0, though mandatory for ARMv8.1 and later. The
use of the crc32 instructions is about ten times as fast as the
software braided calculation of the CRC-32. This can noticeably
speed up the decompression of gzip streams.
Mark Adler [Thu, 3 Jan 2019 02:10:40 +0000 (18:10 -0800)]
Don't bother computing check value after successful inflateSync().
inflateSync() is used to skip invalid deflate data, which means
that the check value that was being computed is no longer useful.
This commit turns off the check value computation, and furthermore
allows a successful return if the compressed data terminated in a
graceful manner. This commit also fixes a bug in the case that
inflateSync() is used before a header is ever processed. In that
case, there is no knowledge of a trailer, so the remainder is
treated as raw.
Mark Adler [Tue, 11 Dec 2018 09:11:38 +0000 (01:11 -0800)]
Speed up software CRC-32 computation by a factor of 1.5 to 3.
Use the interleaved method of Kadatch and Jenkins in order to make
use of pipelined instructions through multiple ALUs in a single
core. This also speeds up and simplifies the combination of CRCs,
and updates the functions to pre-calculate and use an operator for
CRC combination.
Mark Adler [Sun, 4 Nov 2018 18:31:46 +0000 (10:31 -0800)]
Add crc32_combine_gen() and crc32_combine_op() for fast combines.
When the same len2 is used repeatedly, it is faster to use
crc32_combine_gen() to generate an operator, that is then used to
combine CRCs with crc32_combine_op().
Mark Adler [Sat, 4 Aug 2018 21:27:02 +0000 (14:27 -0700)]
Clarify that prefix codes are counted in enough.c.
There is no assurance that all prefix codes are reachable as
optimal Huffman codes for the numbers of symbols encountered in
a deflate block. This code considers all possible prefix codes,
which might be a larger set than all possible Huffman codes,
depending on the constraints.
Mark Adler [Wed, 18 Apr 2018 05:09:22 +0000 (22:09 -0700)]
Fix a bug that can crash deflate on some input when using Z_FIXED.
This bug was reported by Danilo Ramos of Eideticom, Inc. It has
lain in wait 13 years before being found! The bug was introduced
in zlib 1.2.2.2, with the addition of the Z_FIXED option. That
option forces the use of fixed Huffman codes. For rare inputs with
a large number of distant matches, the pending buffer into which
the compressed data is written can overwrite the distance symbol
table which it overlays. That results in corrupted output due to
invalid distances, and can result in out-of-bound accesses,
crashing the application.
The fix here combines the distance buffer and literal/length
buffers into a single symbol buffer. Now three bytes of pending
buffer space are opened up for each literal or length/distance
pair consumed, instead of the previous two bytes. This assures
that the pending buffer cannot overwrite the symbol table, since
the maximum fixed code compressed length/distance is 31 bits, and
since there are four bytes of pending space for every three bytes
of symbol space.
Mark Adler [Sat, 3 Jun 2017 16:49:39 +0000 (09:49 -0700)]
Avoid the use of ptrdiff_t.
This isn't the right type anyway to assure that it contains a
pointer. That type would be intptr_t or uintptr_t. However the C99
standard says that those types are optional, so their use would not
be portable. This commit simply uses size_t or whatever configure
decided to use for size_t. That would be the same length as
ptrdiff_t, and so will work just as well. The code checks to see if
the length of the type used is the same as the length of a void
pointer, so there is already protection against the use of the
wrong type. The use of size_t (or ptrdiff_t) will almost always
work, as all modern architectures have an array size that is the
same as the pointer size. Only old segmented architectures would
have to fall back to the slower CRC-32 calculation, where the
amount of memory that can be accessed is larger than the maximum
array size.
Mark Adler [Sun, 16 Apr 2017 15:35:33 +0000 (08:35 -0700)]
Handle case where inflateSync used when header never processed.
If zlib and/or gzip header processing was requested, but a header
was never provided and inflateSync was used successfully, then the
inflate state would be inconsistent, trying to compute a check
value but with no flags set. This commit sets the inflate mode to
raw in this case, since there is no other assumption that can be
made if a header was requested but never seen.
Mark Adler [Sun, 5 Feb 2017 07:58:37 +0000 (23:58 -0800)]
Avoid a conversion error in gzseek when off_t type too small.
This is a problem in the odd case that the second argument of
LSEEK is a larger type than off_t. Apparently MinGW defines off_t
to be 32 bits, but _lseeki64 has a 64-bit second argument.
Also undo a previous commit to permit MinGW to use _lseeki64.
Mark Adler [Sat, 21 Jan 2017 09:50:26 +0000 (01:50 -0800)]
Limit hash table inserts after switch from stored deflate.
This limits hash table inserts to the available data in the window
and to the sliding window size in deflate_stored(). The hash table
inserts are deferred until deflateParams() switches to a non-zero
compression level.
Mark Adler [Mon, 16 Jan 2017 17:49:35 +0000 (09:49 -0800)]
Permit a deflateParams() parameter change as soon as possible.
This commit allows a parameter change even if the input data has
not all been compressed and copied to the application output
buffer, so long as all of the input data has been compressed to
the internal pending output buffer. This also allows an immediate
deflateParams change so long as there have been no deflate calls
since initialization or reset.
Mark Adler [Sun, 15 Jan 2017 16:22:16 +0000 (08:22 -0800)]
Permit immediate deflateParams changes before any deflate input.
This permits deflateParams to change the strategy and level right
after deflateInit, without having to wait until a header has been
written. The parameters can be changed immediately up until the
first deflate call that consumes any input data.
Mark Adler [Sun, 15 Jan 2017 16:15:55 +0000 (08:15 -0800)]
Update high water mark in deflate_stored.
This avoids unnecessary filling of bytes in the sliding window
buffer when switching from level zero to a non-zero level. This
also provides a consistent indication of deflate having taken
input for a later commit ...
Mark Adler [Tue, 3 Jan 2017 01:25:27 +0000 (17:25 -0800)]
Add warnings when compiling with assembler code.
There have been many reports of bugs in the assembler codes
intended to speed up deflate and inflate. They are third-party
contributions in contrib, and so are not supported by the zlib
maintainers.
Mark Adler [Sat, 31 Dec 2016 18:03:09 +0000 (10:03 -0800)]
Avoid the need for ssize_t.
Limit read() and write() requests to sizes that fit in an int.
This allows storing the return value in an int, and avoiding the
need to use or construct an ssize_t type. This is required for
Microsoft C, whose _read and _write functions take an unsigned
request and return an int.