bors [Sun, 18 May 2014 21:41:35 +0000 (14:41 -0700)]
auto merge of #14000 : pnkfelix/rust/fsk-fix-issue13732, r=alexcrichton
Fix #13732.
This is a revised, much less hacky form of PR #13753
The changes here:
* add instrumentation to aid debugging of linkage errors,
* fine tune some things in the Makefile where we are telling binaries to use a host-oriented path for finding dynamic libraries, when it should be feeding the binaries a target-oriented path for dynamic libraries.
* pass along the current stage number to run-make tests, and
* skip certain tests when running atop stage1.
Refactoring: Introduce distinct host and target rpath var setters.
Two line summary: Distinguish HOST_RPATH and TARGET_RPATH; added
RPATH_LINK_SEARCH; skip tests broken in stage1; general cleanup.
`HOST_RPATH_VAR$(1)_T_$(2)_H_$(3)` and `TARGET_RPATH_VAR$(1)_T_$(2)_H_$(3)`
both match the format of the old `RPATH_VAR$(1)_T_$(2)_H_$(3)` (which
is still being set the same way that it was before, to one of either
HOST/TARGET depending on what stage we are building). Namely, the format
is <XXX>_RPATH_VAR = "<LD_LIB_PATH_ENVVAR>=<COLON_SEP_PATH_ENTRIES>"
What this commit does:
* Pass both of the (newly introduced) HOST and TARGET rpath setup vars
to `maketest.py`
* Update `maketest.py` to no longer update the LD_LIBRARY_PATH itself
Instead, it passes along the HOST and TARGET rpath setup vars in
environment variables `HOST_RPATH_ENV` and `TARGET_RPATH_ENV`
* Also, pass the current stage number to maketest.py; it in turn
passes it (via an env var) to run-make tests.
This allows the run-make tests to selectively change behavior
(e.g. turn themselves off) to deal with incompatibilities with
e.g. stage1.
* Cleanup: Distinguish in tools.mk between the command to run (`RUN`)
and the file to generate to drive that command (`RUN_BINFILE`). The
main thing this enables is that `RUN` can now setup the
`TARGET_RPATH_ENV` without having to dirty up the runner code in
each of the `run-make` Makefiles.
* Cleanup: Factored out commands to delete dylib/rlib into
REMOVE_DYLIBS/REMOVE_RLIBS.
There were places where we were only calling `rm $(call DYLIB,foo)`
even though we really needed to get rid of the whole glob (at least
based on alex's findings on #13753 that removing the symlink does not
suffice).
Therefore rather than peppering the code with the awkward
`rm $(TMPDIR)/$(call DYLIB_GLOB,foo)`, I instead introduced a common
`REMOVE_DYLIBS` user function that expands into that when called.
After I adding an analogous `REMOVE_RLIBS`, I changed all of the
existing calls that rm dylibs or rlibs to use these routines
instead.
Note that the latter is not a true refactoring since I may have
changed cases where it was our intent to only remove the sym-link.
(But if that is the case, then we need to more deeply investigate
alex's findings on #13753 where the system was still dynamically
loading up the non-symlinked libraries that it finds on the load
path.)
* Added RPATH_LINK_SEARCH command and use it on Linux.
On some platforms, namely Linux, when you have libboot.so that has
its internal rpath set (to e.g. $(ORIGIN)/path/to/HOSTDIR), the
linker still complains when you do the link step and it does not
know where to find libraries that libboot.so depends upon that live
in HOSTDIR (think e.g. librustuv.so).
As far as I can tell, the GNU linker will consult the
LD_LIBRARY_PATH as part of the linking process to find such
libraries. But if you want to be more careful and not override
LD_LIBRARY_PATH for the `gcc` invocation, then you need some other
way to tell the linker where it can find the libraries that
libboot.so needs. The solution to this on Linux is the
`-Wl,-rpath-link` command line option.
However, this command line option does not exist on Mac OS X, (which
appears to be figuring out how to resolve the libboot.dylib
dependency by some other means, perhaps by consulting the rpath
setting within libboot.dylib).
So, in order to abstract over this distinction, I added the
RPATH_LINK_SEARCH macro to the run-make infrastructure and added
calls to it where necessary to get Linux working. On architectures
other than Linux, the macro expands to nothing.
* Disable miscellaneous tests atop stage1.
* An especially interesting instance of the previous bullet point:
Excuse regex from doing rustdoc tests atop stage1.
This was a (nearly-) final step to get `make check-stage1` working
again.
The use of a special-case check for regex here is ugly but is
analogous other similar checks for regex such as the one that landed
in PR #13844.
The way this is written, the user will get a reminder that
doc-crate-regex is being skipped whenever their rules attempt to do
the crate documentation tests. This is deliberate: I want people
running `make check-stage1` to be reminded about which cases are
being skipped. (But if such echo noise is considered offensive, it
can obviously be removed.)
* Got windows working with the above changes.
This portion of the commit is a cleanup revision of the (previously
mentioned on try builds) re-architecting of how the LD_LIBRARY_PATH
setup and extension is handled in order to accommodate Windows' (1.)
use of `$PATH` for that purpose and (2.) use of spaces in `$PATH`
entries (problematic for make and for interoperation with tools at
the shell).
* In addition, since the code has been rearchitected to pass the
HOST_RPATH_DIR/TARGET_RPATH_DIR rather than a whole sh
environment-variable setting command, there is no need to for the
convert_path_spec calls in maketest.py, which in fact were put in
place to placate Windows but were now causing the Windows builds to
fail. Instead we just convert the paths to absolute paths just like
all of the other path arguments.
Also, note for makefile hackers: apparently you cannot quote operands
to `ifeq` in Makefile (or at least, you need to be careful about
adding them, e.g. to only one side).
Fix was originally authored by alexcrichton and then rebased a couple
times by pnkfelix, most recently atop PR 13954.
----
Regarding the change to librustdoc/lib.rs, to do `map_err` before
unwrapping a `TqskResult`: I do not understand how master is passing
without this change or something like it, since `Box<Any:Send>` does
not implement `Show`. (Is this something that is only a problem for
the snapshot stage0 compiler?) Still, the change I have put in here
(which was added as part of a rebase after alex's review) seems
harmless to me to apply to rustdoc at all stages, since a call to
`unwrap` is just going to `fail!` on the err case anyway.
bors [Sun, 18 May 2014 20:06:30 +0000 (13:06 -0700)]
auto merge of #14275 : kballard/rust/bytes-return-static, r=huonw
Change `bytes!()` to return
{
static BYTES: &'static [u8] = &[...];
BYTES
}
This gives it the `'static` lifetime, whereas before it had an rvalue
lifetime. Until recently this would have prevented assigning `bytes!()`
to a static, as in
static FOO: &'static [u8] = bytes!(1,2,3);
but #14183 fixed it so blocks are now allowed in constant expressions
(with restrictions).
Kevin Ballard [Sat, 18 Jan 2014 06:30:36 +0000 (22:30 -0800)]
Make bytes!() return 'static
Change `bytes!()` to return
{
static BYTES: &'static [u8] = &[...];
BYTES
}
This gives it the `'static` lifetime, whereas before it had an rvalue
lifetime. Until recently this would have prevented assigning `bytes!()`
to a static, as in
static FOO: &'static [u8] = bytes!(1,2,3);
but #14183 fixed it so blocks are now allowed in constant expressions
(with restrictions).
bors [Sun, 18 May 2014 09:51:30 +0000 (02:51 -0700)]
auto merge of #14258 : alexcrichton/rust/dox-format-writer, r=cmr
This commit fills in the documentation holes for the FormatWriter trait which
were previously accidentally left blank. Additionally, this adds the `write_fmt`
method to the trait to allow usage of the `write!` macro with implementors of
the `FormatWriter` trait. This is not useful for consumers of the standard
library who should generally avoid the `FormatWriter` trait, but it is useful
for consumers of the core library who are not using the standard library.
bors [Sun, 18 May 2014 08:16:27 +0000 (01:16 -0700)]
auto merge of #14121 : luqmana/rust/option-ffi, r=alexcrichton
This slightly adjusts the NullablePointer representation for some enums in the case where the non-nullable variant has a single field (the ptr field) to be just that, the pointer. This is in contrast to the current behaviour where we'd wrap that single pointer in a LLVM struct.
Alex Crichton [Sat, 17 May 2014 05:40:38 +0000 (22:40 -0700)]
core: Document FormatWriter and allow `write!`
This commit fills in the documentation holes for the FormatWriter trait which
were previously accidentally left blank. Additionally, this adds the `write_fmt`
method to the trait to allow usage of the `write!` macro with implementors of
the `FormatWriter` trait. This is not useful for consumers of the standard
library who should generally avoid the `FormatWriter` trait, but it is useful
for consumers of the core library who are not using the standard library.
bors [Sun, 18 May 2014 05:01:28 +0000 (22:01 -0700)]
auto merge of #14230 : alexcrichton/rust/liballoc, r=cmr,huonw
This commit is part of the libstd facade RFC, issue #13851. This creates a new
library, liballoc, which is intended to be the core allocation library for all
of Rust. It is pinned on the basic assumption that an allocation failure is an
abort or failure.
This module has inherited the heap/libc_heap modules from std::rt, the owned/rc
modules from std, and the arc module from libsync. These three pointers are
currently the three most core pointer implementations in Rust.
The UnsafeArc type in std::sync should be considered deprecated and replaced by
`Arc<Unsafe<T>>`. This commit does not currently migrate to this type, but future
commits will continue this refactoring.
Alex Crichton [Tue, 13 May 2014 23:10:05 +0000 (16:10 -0700)]
std: Refactor liballoc out of lib{std,sync}
This commit is part of the libstd facade RFC, issue #13851. This creates a new
library, liballoc, which is intended to be the core allocation library for all
of Rust. It is pinned on the basic assumption that an allocation failure is an
abort or failure.
This module has inherited the heap/libc_heap modules from std::rt, the owned/rc
modules from std, and the arc module from libsync. These three pointers are
currently the three most core pointer implementations in Rust.
The UnsafeArc type in std::sync should be considered deprecated and replaced by
Arc<Unsafe<T>>. This commit does not currently migrate to this type, but future
commits will continue this refactoring.
bors [Sat, 17 May 2014 12:06:27 +0000 (05:06 -0700)]
auto merge of #14253 : alexcrichton/rust/issue-14221, r=pcwalton
This plugs a leak where resolve was treating enums defined in parent modules as
in-scope for all children modules when resolving a pattern identifier. This
eliminates the code path in resolve entirely.
If this breaks any existing code, then it indicates that the variants need to be
explicitly imported into the module.
bors [Fri, 16 May 2014 23:31:24 +0000 (16:31 -0700)]
auto merge of #14216 : kballard/rust/macos_precise_time_ns, r=alexcrichton
Use sync::one::Once to fetch the mach_timebase_info only once when
running precise_time_ns(). This helps because mach_timebase_info() is
surprisingly inefficient. Also fix the order of operations when applying
the timebase to the mach absolute time value.
This improves the time on my machine from
```
test tests::bench_precise_time_ns ... bench: 157 ns/iter (+/- 4)
```
to
```
test tests::bench_precise_time_ns ... bench: 38 ns/iter (+/- 3)
```
Alex Crichton [Fri, 16 May 2014 22:44:14 +0000 (15:44 -0700)]
rustc: Stop leaking enum variants into children
This plugs a leak where resolve was treating enums defined in parent modules as
in-scope for all children modules when resolving a pattern identifier. This
eliminates the code path in resolve entirely.
If this breaks any existing code, then it indicates that the variants need to be
explicitly imported into the module.
bors [Fri, 16 May 2014 21:46:24 +0000 (14:46 -0700)]
auto merge of #14135 : gereeter/rust/two-way-search, r=brson
This changes the previously naive string searching algorithm to a two-way search like glibc, which should be faster on average while still maintaining worst case linear time complexity. This fixes #14107. Note that I don't think this should be merged yet, as this is the only approach to speeding up search I've tried - it's worth considering options like Boyer-Moore or adding a bad character shift table to this. However, the benchmarks look quite good so far:
test str::bench::bench_contains_bad_naive ... bench: 290 ns/iter (+/- 12) from 1309 ns/iter (+/- 36)
test str::bench::bench_contains_equal ... bench: 479 ns/iter (+/- 10) from 137 ns/iter (+/- 2)
test str::bench::bench_contains_short_long ... bench: 2844 ns/iter (+/- 105) from 5473 ns/iter (+/- 14)
test str::bench::bench_contains_short_short ... bench: 55 ns/iter (+/- 4) from 57 ns/iter (+/- 6)
Except for the case specifically designed to be optimal for the naive case (`bench_contains_equal`), this gets as good or better performance as the previous code.
Kevin Ballard [Thu, 15 May 2014 00:54:36 +0000 (17:54 -0700)]
Optimize and fix time::precise_time_ns() on macos
Use sync::one::Once to fetch the mach_timebase_info only once when
running precise_time_ns(). This helps because mach_timebase_info() is
surprisingly inefficient. Also fix the order of operations when applying
the timebase to the mach absolute time value.
This improves the time on my machine from
```
test tests::bench_precise_time_ns ... bench: 157 ns/iter (+/- 4)
```
to
```
test tests::bench_precise_time_ns ... bench: 38 ns/iter (+/- 3)
```
bors [Fri, 16 May 2014 09:46:25 +0000 (02:46 -0700)]
auto merge of #14237 : alexcrichton/rust/issue-14144, r=cmr
By default, jemalloc is building itself with -g3 if the local compiler supports
it. It looks like this is generating a good deal of debug info that windows
isn't optimizing out (on the order of 18MB). Windows gcc/ld is also not
optimizing this data away, causing hello world to be 18MB in size.
There's no current real need for debugging jemalloc to a great extent, so this
commit manually passes -g1 to override -g3 which jemalloc is using. This is
confirmed to drop the size of executables on windows back to a more reasonable
size (2.0MB, as they were before).
bors [Fri, 16 May 2014 06:26:26 +0000 (23:26 -0700)]
auto merge of #14115 : alexcrichton/rust/core-fmt, r=brson
This was a more difficult change than I thought it would be, and it is unfortunately a breaking change rather than a drop-in replacement. Most of the rationale can be found in the third commit.
Alex Crichton [Sat, 10 May 2014 21:19:26 +0000 (14:19 -0700)]
std: Delegate some integer formatting to core::fmt
In an attempt to phase out the std::num::strconv module's string formatting
functionality, this commit reimplements some provided methods for formatting
integers on top of format!() instead of the custom (and slower) implementation
inside of num::strconv.
Primarily, this deprecates int_to_str_bytes_common
Alex Crichton [Sat, 10 May 2014 21:06:25 +0000 (14:06 -0700)]
core: Implement f32/f64 formatting
This is a migration of the std::{f32, f64}::to_str* functionality to the core
library. This removes the growable `Vec` used in favor of a large stack buffer.
The maximum base 10 exponent for f64 is 308, so a stack buffer of 512 bytes
should be sufficient to store all floats.
Alex Crichton [Sat, 10 May 2014 21:05:06 +0000 (14:05 -0700)]
Updates with core::fmt changes
1. Wherever the `buf` field of a `Formatter` was used, the `Formatter` is used
instead.
2. The usage of `write_fmt` is minimized as much as possible, the `write!` macro
is preferred wherever possible.
3. Usage of `fmt::write` is minimized, favoring the `write!` macro instead.
Alex Crichton [Sat, 10 May 2014 20:53:40 +0000 (13:53 -0700)]
syntax: Add a macro, format_args_method!()
Currently, the format_args!() macro takes as its first argument an expression
which is the callee of an ExprCall. This means that if format_args!() is used
with calling a method a closure must be used. Consider this code, however:
The closure borrows the entire `foo` structure, disallowing the later borrow of
`foo.field`. To preserve the semantics of the `write!` macro, it is also
impossible to borrow specifically the `writer` field of the `foo` structure
because it must be borrowed mutably, but the `foo` structure is not guaranteed
to be mutable itself.
This macro will generate an ExprMethodCall which allows the borrow checker to
understand that `writer` and `field` should be borrowed separately.
This macro is not strictly necessary, with DST or possibly UFCS other
workarounds could be used. For now, though, it looks like this is required to
implement the `write!` macro.
Alex Crichton [Sat, 10 May 2014 20:52:26 +0000 (13:52 -0700)]
std: Add an adaptor for Writer => FormatWriter
This new method, write_fmt(), is the one way to write a formatted list of
arguments into a Writer stream. This has a special adaptor to preserve errors
which occur on the writer.
All macros will be updated to use this method explicitly.
Alex Crichton [Sat, 10 May 2014 20:33:43 +0000 (13:33 -0700)]
core: Inherit the std::fmt module
This commit moves all possible functionality from the standard library's string
formatting utilities into the core library. This is a breaking change, due to a
few tweaks in the semantics of formatting:
1. In order to break the dependency on the std::io module, a new trait,
FormatWriter was introduced in core::fmt. This is the trait which is used
(instead of Writer) to format data into a stream.
2. The new FormatWriter trait has one method, write(), which takes some bytes
and can return an error, but the error contains very little information. The
intent for this trait is for an adaptor writer to be used around the standard
library's Writer trait.
3. The fmt::write{,ln,_unsafe} methods no longer take &mut io::Writer, but
rather &mut FormatWriter. Since this trait is less common, all functions were
removed except fmt::write, and it is not intended to be invoked directly.
The main API-breaking change here is that the fmt::Formatter structure will no
longer expose its `buf` field. All previous code writing directly to `f.buf`
using writer methods or the `write!` macro will now instead use `f` directly.
The Formatter object itself implements the `Writer` trait itself for
convenience, although it does not implement the `FormatWriter` trait. The
fallout of these changes will be in the following commits.
Alex Crichton [Sat, 10 May 2014 20:25:49 +0000 (13:25 -0700)]
core: Move intrinsic float functionality from std
The Float trait in libstd is quite a large trait which has dependencies on cmath
(libm) and such, which libcore cannot satisfy. It also has many functions that
libcore can implement, however, as LLVM has intrinsics or they're just bit
twiddling.
This commit moves what it can of the Float trait from the standard library into
libcore to allow floats to be usable in the core library. The remaining
functions are now resident in a FloatMath trait in the standard library (in the
prelude now). Previous code which was generic over just the Float trait may now
need to be generic over the FloatMath trait.
bors [Fri, 16 May 2014 00:36:27 +0000 (17:36 -0700)]
auto merge of #14196 : chris-morgan/rust/hashmap-mangle, r=cmr
This used to be called `mangle` and was removed when the Robin Hood hash map came along, but it is a useful thing to have in certain situations (I just hit it with my Teepee header representation), so I want it back.
The method is renamed to `find_with_or_insert_with`, also with the parameters swapped to make sense—find and then insert, not insert and then find.
Alex Crichton [Thu, 15 May 2014 22:43:40 +0000 (15:43 -0700)]
mk: Don't build jemalloc with -g3
By default, jemalloc is building itself with -g3 if the local compiler supports
it. It looks like this is generating a good deal of debug info that windows
isn't optimizing out (on the order of 18MB). Windows gcc/ld is also not
optimizing this data away, causing hello world to be 18MB in size.
There's no current real need for debugging jemalloc to a great extent, so this
commit manually passes -g1 to override -g3 which jemalloc is using. This is
confirmed to drop the size of executables on windows back to a more reasonable
size (2.0MB, as they were before).
Alex Crichton [Thu, 15 May 2014 21:59:01 +0000 (14:59 -0700)]
Test fixes from rollup
Closes #14231 (mk: Don't run benchmarks with `make check`)
Closes #14215 (std: Modify TempDir to not fail on drop. Closes #12628)
Closes #14211 (rustdoc: functions in ffi blocks are unsafe)
Closes #14210 (Make Vec.truncate() resilient against failure in Drop)
Closes #14208 (Make `from_bits` in `bitflags!` safe; add `from_bits_truncate`)
Closes #14206 (Register new snapshots)
Closes #14205 (use sched_yield on linux and freebsd)
Closes #14204 (Add a crate for missing stubs from libcore)
Closes #14203 (shootout-mandelbrot: Either 10-20% or 80-100% improvement.)
Closes #14202 (Add flow-graph visualization (via graphviz) to rustc)
Closes #14201 (Render not_found with an absolute path to the rust stylesheet)
Closes #14200 (std cleanup)
Closes #14189 (Implement cell::clone_ref)
Each test works by rendering the flowgraph for the last identified
block we see in expanded pretty-printed output, and comparing it (via
`diff`) against a checked in "foo.dot-expected.dot" file.
Each test post-processes the output to remove NodeIds ` (id=NUM)` so
that the expected output is somewhat stable (or at least independent
of how we assign NodeIds) and easier for a human to interpret when
looking at the expected output file itself.
----
Test writing style notes:
I usually tried to write the tests in a way that would avoid duplicate
labels in the output rendered flow graph, when possible.
The tests that have string literals "unreachable" in the program text
are deliberately written that way to remind the reader that the
unreachable nodes in the resulting graph are not an error in the
control flow computation, but rather a natural consequence of its
construction.
Passing `--pretty flowgraph=<NODEID>` makes rustc print a control flow graph.
In pratice, you will also need to pass the additional option:
`-o <FILE>` to emit output to a `.dot` file for graphviz.
(You can only print the flow-graph for a particular block in the AST.)
----
An interesting implementation detail is the way the code puts both the
node index (`cfg::CFGIndex`) and a reference to the payload
(`cfg::CFGNode`) into the single `Node` type that is used for
labelling and walking the graph. I had once mistakenly thought that I
only wanted the `cfg::CFGNode`, but for labelling, you really want the
cfg index too, rather than e.g. trying to use the `ast::NodeId` as the
label (which breaks down e.g. due to `ast::DUMMY_NODE_ID`).
----
As a drive-by fix, I had to fix `rustc::middle::cfg::construct`
interface to reflect changes that have happened on the master branch
while I was getting this integrated into the compiler. (The next
commit actually adds tests of the `--pretty flowgraph` functionality,
so that should ensure that the `rustc::middle::cfg` code does not go
stale again.)
2. Revise handling of `break` and `continue` forms so that they are
not treated as if control falls through to the next node (since it
does not, it just jumps to the end or start of the loop body).
3. Fixed support for return expression in flow graph construction.
Refine lifetimes in signature for graph node/edge iteration methods.
Added `pub` `node_id` and `edge_id` methods that correspond to
NodeIndex and EdgeIndex `get` methods (note that the inner index is
already `pub` in the struct definitions). (I decided that `get()`,
used internally, just looks too generic and that client code is
clearer with more explicit method names.)
Add `EntryPat` and `NodePat` variants to ast_map, so that lookups for
id 1 in `let S{val: _x /* pat 2 */} /* pat 1 */ = ...` will actually
resolve to the pattern `S{ ... }`, rather than "unknown node", in a
function like `node_id_to_str`.