Mark Adler [Sun, 15 Jan 2017 16:22:16 +0000 (08:22 -0800)]
Permit immediate deflateParams changes before any deflate input.
This permits deflateParams to change the strategy and level right
after deflateInit, without having to wait until a header has been
written. The parameters can be changed immediately up until the
first deflate call that consumes any input data.
Mark Adler [Sun, 15 Jan 2017 16:15:55 +0000 (08:15 -0800)]
Update high water mark in deflate_stored.
This avoids unnecessary filling of bytes in the sliding window
buffer when switching from level zero to a non-zero level. This
also provides a consistent indication of deflate having taken
input for a later commit ...
Mark Adler [Tue, 3 Jan 2017 01:25:27 +0000 (17:25 -0800)]
Add warnings when compiling with assembler code.
There have been many reports of bugs in the assembler codes
intended to speed up deflate and inflate. They are third-party
contributions in contrib, and so are not supported by the zlib
maintainers.
Mark Adler [Sat, 31 Dec 2016 18:03:09 +0000 (10:03 -0800)]
Avoid the need for ssize_t.
Limit read() and write() requests to sizes that fit in an int.
This allows storing the return value in an int, and avoiding the
need to use or construct an ssize_t type. This is required for
Microsoft C, whose _read and _write functions take an unsigned
request and return an int.
Mark Adler [Sat, 3 Dec 2016 18:27:14 +0000 (10:27 -0800)]
Create z_size_t and z_ssize_t types.
Normally these are set to size_t and ssize_t. But if they do not
exist, then they are set to the smallest integer type that can
contain a pointer. size_t is unsigned and ssize_t is signed.
Mark Adler [Sat, 3 Dec 2016 16:29:57 +0000 (08:29 -0800)]
Don't need to emit an empty fixed block when changing parameters.
gzsetparams() was using Z_PARTIAL_FLUSH when it could use Z_BLOCK
instead. This commit uses Z_BLOCK, which avoids emitting an
unnecessary ten bits into the stream.
Mark Adler [Sat, 3 Dec 2016 16:18:56 +0000 (08:18 -0800)]
Clean up gz* function return values.
In some cases the return values did not match the documentation,
or the documentation did not document all of the return values.
gzprintf() now consistently returns negative values on error,
which matches the behavior of the stdio fprintf() function.
Mark Adler [Sat, 5 Nov 2016 15:43:29 +0000 (08:43 -0700)]
Speed up deflation for level 0 (storing).
The previous code slid the window and the hash table and copied
every input byte three times in order to just write the data as
stored blocks with no compression. This commit minimizes sliding
and copying, especially for large input and output buffers.
Level 0 compression is now more than 20 times faster than before
the commit.
Most of the speedup is due to deferring hash table slides until
deflateParams() is called to change the compression level away
from 0. More speedup is due to copying directly from next_in to
next_out when the amounts of available input data and output space
permit it, avoiding the intermediate pending buffer. Additionally,
only the last 32K of the used input data is copied back to the
sliding window when large input buffers are provided.
Mark Adler [Wed, 23 Nov 2016 07:29:19 +0000 (23:29 -0800)]
Assure that deflateParams() will not switch functions mid-block.
This alters the specification in zlib.h, so that deflateParams()
will not change any parameters if there is not enough output space
in the event that a block is emitted in order to allow switching
the compression function.
Mark Adler [Sun, 20 Nov 2016 19:36:15 +0000 (11:36 -0800)]
Increase verbosity required to warn about bit length overflow.
When debugging the Huffman coding would warn about resulting codes
greater than 15 bits in length. This is handled properly, and is
not uncommon. This increases the verbosity of the warning by one,
so that it is not displayed by default.
Mark Adler [Sun, 30 Oct 2016 16:25:32 +0000 (09:25 -0700)]
Use memcpy for stored blocks.
This speeds up level 0 by about a factor of three, as compared to
the previous byte-at-a-time loop. We can do much better though. A
later commit avoids this copy for level 0 with large buffers,
instead copying directly from the input to the output. This commit
still speeds up storing incompressible data found when compressing
normally.
Mark Adler [Fri, 28 Oct 2016 05:50:43 +0000 (22:50 -0700)]
Fix bug when level 0 used with Z_HUFFMAN or Z_RLE.
Compression level 0 requests no compression, using only stored
blocks. When Z_HUFFMAN or Z_RLE was used with level 0 (granted,
an odd choice, but permitted), the resulting blocks were mostly
fixed or dynamic. The reason is that deflate_stored() was not
being called in that case. The compressed data was valid, but it
was not what the application requested. This commit assures that
only stored blocks are emitted for compression level 0, regardless
of the strategy selected.
Mark Adler [Wed, 26 Oct 2016 03:45:41 +0000 (20:45 -0700)]
Make a noble effort at setting OS_CODE correctly.
This updates the OS_CODE determination at compile time to match as
closely as possible the operating system mappings documented in
the PKWare APPNOTE.TXT version 6.3.4, section 4.4.2.2. That byte
in the gzip header is used by nobody for anything, as far as I can
tell. However we might as well try to set it appropriately.
Mark Adler [Tue, 25 Oct 2016 03:11:41 +0000 (20:11 -0700)]
Do a more thorough check of the state for every stream call.
This verifies that the state has been initialized, that it is the
expected type of state, deflate or inflate, and that at least the
first several bytes of the internal state have not been clobbered.
Mark Adler [Mon, 24 Oct 2016 22:52:19 +0000 (15:52 -0700)]
Reject a window size of 256 bytes if not using the zlib wrapper.
There is a bug in deflate for windowBits == 8 (256-byte window).
As a result, zlib silently changes a request for 8 to a request
for 9 (512-byte window), and sets the zlib header accordingly so
that the decompressor knows to use a 512-byte window. However if
deflateInit2() is used for raw deflate or gzip streams, then there
is no indication that the request was not honored, and the
application might assume that it can use a 256-byte window when
decompressing. This commit returns an error if the user requests
a 256-byte window when using raw deflate or gzip encoding.
Mark Adler [Tue, 4 Oct 2016 05:33:26 +0000 (22:33 -0700)]
Note the violation of the strict aliasing rule in crc32.c.
See the comment for more details. This is in response to an issue
raised as a result of a security audit of the zlib code by Trail
of Bits and TrustInSoft, in support of the Mozilla Foundation.
Mark Adler [Thu, 29 Sep 2016 03:20:25 +0000 (20:20 -0700)]
Avoid pre-decrement of pointer in big-endian CRC calculation.
There was a small optimization for PowerPCs to pre-increment a
pointer when accessing a word, instead of post-incrementing. This
required prefacing the loop with a decrement of the pointer,
possibly pointing before the object passed. This is not compliant
with the C standard, for which decrementing a pointer before its
allocated memory is undefined. When tested on a modern PowerPC
with a modern compiler, the optimization no longer has any effect.
Due to all that, and per the recommendation of a security audit of
the zlib code by Trail of Bits and TrustInSoft, in support of the
Mozilla Foundation, this "optimization" was removed, in order to
avoid the possibility of undefined behavior.
Mark Adler [Thu, 22 Sep 2016 06:35:50 +0000 (23:35 -0700)]
Remove offset pointer optimization in inftrees.c.
inftrees.c was subtracting an offset from a pointer to an array,
in order to provide a pointer that allowed indexing starting at
the offset. This is not compliant with the C standard, for which
the behavior of a pointer decremented before its allocated memory
is undefined. Per the recommendation of a security audit of the
zlib code by Trail of Bits and TrustInSoft, in support of the
Mozilla Foundation, this tiny optimization was removed, in order
to avoid the possibility of undefined behavior.
Mark Adler [Thu, 22 Sep 2016 05:25:21 +0000 (22:25 -0700)]
Use post-increment only in inffast.c.
An old inffast.c optimization turns out to not be optimal anymore
with modern compilers, and furthermore was not compliant with the
C standard, for which decrementing a pointer before its allocated
memory is undefined. Per the recommendation of a security audit of
the zlib code by Trail of Bits and TrustInSoft, in support of the
Mozilla Foundation, this "optimization" was removed, in order to
avoid the possibility of undefined behavior.
Mark Adler [Thu, 22 Sep 2016 03:07:37 +0000 (20:07 -0700)]
Remove dummy structure declarations for old buggy compilers.
While woolly mammoths still roamed the Earth and before Atlantis
sunk into the ocean, there were C compilers that could not handle
forward structure references, e.g. "struct name;". zlib dutifully
provided a work-around for such compilers. That work-around is no
longer needed, and, per the recommendation of a security audit of
the zlib code by Trail of Bits and TrustInSoft, in support of the
Mozilla Foundation, should be removed since what a compiler will
do with this is technically undefined. From the report: "there is
no telling what interactions the bug could have in the future with
link-time optimizations and type-based alias analyses, both
features that are present (but not default) in clang."
Mark Adler [Wed, 21 Sep 2016 01:49:21 +0000 (18:49 -0700)]
Add option to not compute or check check values.
The undocumented (except in these commit comments) function
inflateValidate(strm, check) can be called after an inflateInit(),
inflateInit2(), or inflateReset2() with check equal to zero to
turn off the check value (CRC-32 or Adler-32) computation and
comparison. Calling with check not equal to zero turns checking
back on. This should only be called immediately after the init or
reset function. inflateReset() does not change the state, so a
previous inflateValidate() setting will remain in effect.
This also turns off validation of the gzip header CRC when
present.
This should only be used when a zlib or gzip stream has already
been checked, and repeated decompressions of the same stream no
longer need to be validated.
Mark Adler [Fri, 27 Nov 2015 06:52:25 +0000 (22:52 -0800)]
Fix bug that accepted invalid zlib header when windowBits is zero.
When windowBits is zero, the size of the sliding window comes from
the zlib header. The allowed values of the four-bit field are
0..7, but when windowBits is zero, values greater than 7 are
permitted and acted upon, resulting in large, mostly unused memory
allocations. This fix rejects such invalid zlib headers.
Mark Adler [Sun, 4 Oct 2015 18:45:00 +0000 (11:45 -0700)]
Use a consistent and more modern approach to not use a parameter.
A remarkably creative and diverse set of approaches to letting the
compiler know that opaque was being used when it wasn't is changed
by this commit to the more standard (void)opaque.
Mark Adler [Sun, 2 Aug 2015 21:46:58 +0000 (14:46 -0700)]
Allow building zlib outside of the source directory.
To build, simply run configure from the source directory by
specifying its path. That path will be used to find the source
files. The source directory will not be touched. All new and
modified files will be made in the current directory. Discovered
in the process that not all makes understand % or $<, and not all
compilers understand -include or -I-. This required a larger
Makefile.in with explicit dependencies.
Mark Adler [Sun, 2 Aug 2015 07:02:07 +0000 (00:02 -0700)]
Align deflateParams() and its documentation in zlib.h.
This updates the documentation to reflect the behavior of
deflateParams() when it is not able to compress all of the input
data provided so far due to insufficient output space. It also
assures that data provided is compressed before the parameter
changes, even if at the beginning of the stream.
Mark Adler [Wed, 29 Jul 2015 04:41:20 +0000 (21:41 -0700)]
Fix inflateInit2() bug when windowBits is 16 or 32.
A windowBits value of 0, 16, or 32 gets the window bits from the
zlib header. However there is no zlib header for 16, or for 32
when the input is gzip. This commit sets the window bits for
inflate to 15 if a gzip stream is detected and windowBits was 16
or 32.