bors [Fri, 12 Feb 2016 05:53:18 +0000 (05:53 +0000)]
Auto merge of #31368 - JohanLorenzo:dont-strip-if-test-build, r=alexcrichton
Tools which rely on DWARF for generating code coverage report, don't generate accurate numbers on test builds. For instance, [this sample main](https://github.com/JohanLorenzo/rust-testing-example/blob/757bdbf3887f43db9771c20cb72dfc32aa8f4321/src/main.rs) returns [100% coverage](https://coveralls.io/builds/4940156/source?filename=main.rs) when [kcov](https://github.com/SimonKagstrom/kcov/) runs.
With @pnkfelix 's great help, we could narrow down the issue: The linker strips unused function during phase 6. Here's a patch which stops stripping when someone calls `rustc --test $ARGS`. @pnkfelix wasn't sure if we should add a new flag, or just use --test. What do you think @alexcrichton ?
Also, I'm not too sure: where is the best place to add a test for this addition?
bors [Fri, 12 Feb 2016 00:19:13 +0000 (00:19 +0000)]
Auto merge of #31123 - alexcrichton:who-doesnt-want-two-build-systems, r=brson
This series of commits adds the initial implementation of a new build system for
the compiler and standard library based on Cargo. The high-level architecture
now looks like:
1. The `./configure` script is run with `--enable-rustbuild` and other standard
configuration options.
2. A `Makefile` is generate which proxies commands to the new build system.
3. The new build system has a Python script entry point which manages
downloading both a Rust and Cargo nightly. This initial script also manages
building the build system itself (which is written in Rust).
4. The build system, written in rust and called `bootstrap`, architects how to
call `cargo` and manages building all native libraries and such.
One might reasonably ask "why rewrite the build system?", which is a good
question! The Rust project has used Makefiles for as long as I can remember at
least, and while ugly and difficult to use are undeniably robust as they contain
years worth of tweaking and tuning for working on as many platforms in as many
situation as possible. The rationale behind this PR, however is:
* The makefiles are impenetrable to all but a few people on this
planet. This means that contributions to the build system are almost
nonexistent, and furthermore if a build system change is needed it's
incredibly difficult to figure out how to do so. This hindrance prevents us
from doing some "perhaps fancier" things we may wish to do in make.
* Our build system, while portable, is unfortunately not infinitely portable
everywhere. For example the recently-introduced MSVC target is quite unlikely
to have `make` installed by default (e.g. it requires building inside of an
MSYS2 shell currently). Conversely, the portability of make comes at a cost of
crazy and weird hacks to work around all sorts of versions of software
everywhere, especially when it comes to the configure script and makefiles.
By rewriting this logic in one of the most robust platforms there is, Rust,
we get to assuage all of these worries for free!
* There's a standard tool to build Rust crates, Cargo, but the standard library
and compiler don't use it. This means that they cannot benefit easily from the
crates.io ecosystem, nor can the ecosystem benefit from a standard way to
build this repository itself. Moving to Cargo should help assuage both of
these needs. This has the added benefit of making the compiler more
approachable for newbies as working on the compiler will just happen to be
working on a large Cargo project, all the same standard tools and tricks will
apply.
* There's a huge amount of portability information in the main distribution, for
example around cross compiling, compiling on new OSes, etc. Pushing this logic
into standard crates (like `gcc`) enables the community to immediately benefit
from new build logic.
Despite these benefits, it's going to be a long road to actually replace our
current build system. This PR is just the beginning and doesn't implement the
full suite of functionality as the current one, but there are many more to
follow! The current implementation strategy hopes to look like:
1. Land a second build system in-tree that can be itereated on an and
contributed to. This will not be used just yet in terms of gating new commits
to the repo.
2. Over time, bring the second build system to feature parity with the old build
system, start setting up CI for both build systems.
3. At some point in the future, switch the default to the new build system, but
keep the old one around.
4. At some further point in the future, delete the entire old build system.
---
Alright, so with all that out of the way, here's some more info on this PR
itself. The inital build system here is contained in the `src/bootstrap`
directory and just adds the necessary minimum bits to bootstrap the compiler
itself. There is currently no support for building documentation, running tests,
or installing, but the implemented support is:
* Compiling LLVM with `cmake` instead of `./configure` + `make`. The LLVM
project is removing their autotools build system, so we'd have to make this
transition eventually anyway.
* Compiling compiler-rt with `cmake` as well (for the same rationale as above).
* Adding `Cargo.toml` to map out the dependency graph to all crates, and also
adding `build.rs` files where appropriate. For example `alloc_jemalloc` has a
script to build jemalloc, `flate` has a script to build `miniz.c`, `std` will
build `libbacktrace`, etc.
* Orchestrating all the calls to `cargo` to build the standard distribution,
following the normal bootstrapping process. This also tracks dependencies
between steps to ensure cross-compilation targets happen as well.
* Configuration is intended to eventually be done through a `config.toml` file,
so support is implemented for this. The most likely vector of configuration
for now, however, is likely through `config.mk` (what `./configure` emits), so
the build system currently parses this information.
There's still quite a few steps left to do, and I'll open up some follow-up
issues (as well as a tracking issue) for this migration, but hopefully this is a
great start to get going! This PR is currently tested on all the
Windows/Linux/OSX triples for x86\_64 and x86, but more portability is always
welcome!
---
Future functionality left to implement
* [ ] Re-verify that multi-host builds work
* [ ] Verify android build works
* [ ] Verify iOS build work (mostly compiler-rt)
* [ ] Verify sha256 and ideally gpg of downloaded nightly compiler and nightly rustc
* [ ] Implement testing -- this is a huge bullet point with lots of sub-bullets
* [ ] Build and generate documentation (plus the various tools we have in-tree)
* [ ] Move various src/etc scripts into Rust -- not sure how this interacts with `make` build system
* [ ] Implement `make install` - like testing this is also quite massive
* [x] Deduplicate version information with makefiles
bors [Thu, 11 Feb 2016 22:22:54 +0000 (22:22 +0000)]
Auto merge of #31545 - dotdash:no_noalias, r=alexcrichton
LLVM's memory dependence analysis doesn't properly account for calls
that could unwind and thus effectively act as a branching point. This
can lead to stores that are only visible when the call unwinds being
removed, possibly leading to calls to drop() functions with b0rked
memory contents.
As there is no fix for this in LLVM yet and we want to keep
compatibility to current LLVM versions anyways, we have to workaround
this bug by omitting the noalias attribute on &mut function arguments.
Benchmarks suggest that the performance loss by this change is very
small.
Thanks to @RalfJung for pushing me towards not removing too many
noalias annotations and @alexcrichton for helping out with the test for
this bug.
Alex Crichton [Fri, 22 Jan 2016 02:37:46 +0000 (18:37 -0800)]
rustc_back: Fix disabling jemalloc
When building with Cargo we need to detect `feature = "jemalloc"` to enable
jemalloc, so propagate this same change to the build system to pass the right
`--cfg` argument.
Alex Crichton [Fri, 22 Jan 2016 01:05:04 +0000 (17:05 -0800)]
configure: Add an option to use the cargo build system
This commit adds a `--enable-rustbuild` option to the configure script which
will copy a different `Makefile.in` into place to intercept all `make`
invocations.
Currently this makefile only has one target, but it's expected to be filled out
quite a bit over time!
Alex Crichton [Thu, 21 Jan 2016 23:36:25 +0000 (15:36 -0800)]
bootstrap: Add directives to not double-link libs
Have all Cargo-built crates pass `--cfg cargobuild` and then add appropriate
`#[cfg]` definitions to all crates to avoid linking anything if this is passed.
This should help allow libstd to compile with both the makefiles and with Cargo.
Alex Crichton [Thu, 21 Jan 2016 23:21:13 +0000 (15:21 -0800)]
bootstrap: Add build scripts for crates
This commits adds build scripts to the necessary Rust crates for all the native
dependencies. This is currently a duplication of the support found in mk/rt.mk
and is my best effort at representing the logic twice, but there may be some
unfortunate-and-inevitable divergence.
As a summary:
* alloc_jemalloc - build script to compile jemallocal
* flate - build script to compile miniz.c
* rustc_llvm - build script to run llvm-config and learn about how to link it.
Note that this crucially (and will not ever) compile LLVM as that would take
far too long.
* rustdoc - build script to compile hoedown
* std - script to determine lots of libraries/linkages as well as compile
libbacktrace
Alex Crichton [Fri, 20 Nov 2015 00:55:21 +0000 (16:55 -0800)]
bootstrap: Read configuration from config.mk
During the transition period where we're still using ./configure and makefiles,
read some extra configuration from `config.mk` if it's present. This means that
the bootstrap build should be configured the same as the original ./configure
invocation.
Eventually this will all be removed in favor of only storing information in
`config.toml` (e.g. the configure script will generate config.toml), but for now
this should suffice.
Alex Crichton [Thu, 19 Nov 2015 23:20:12 +0000 (15:20 -0800)]
Add a Cargo-based build system
This commit is the start of a series of commits which start to replace the
makefiles with a Cargo-based build system. The aim is not to remove the
makefiles entirely just yet but rather just replace the portions that invoke the
compiler to do the bootstrap. This commit specifically adds enough support to
perform the bootstrap (and all the cross compilation within) along with
generating documentation.
More commits will follow up in this series to actually wire up the makefiles to
call this build system, so stay tuned!
bors [Thu, 11 Feb 2016 12:52:42 +0000 (12:52 +0000)]
Auto merge of #31487 - oli-obk:breaking_batch/ast/unop, r=Manishearth
r? @Manishearth
I just noticed they can't be rolled up (often modifying the same line(s) in imports). So once I reach the critical amount for them to be merged I'll create a PR that merges all of them.
bors [Thu, 11 Feb 2016 10:56:45 +0000 (10:56 +0000)]
Auto merge of #31525 - antonblanchard:powerpc64_merge4, r=alexcrichton
We no longer have a separate powerpc64 and powerpc64le target_arch, and instead use target_endian to select between the two. These patches fix a couple of remaining issues.
Johan Lorenzo [Tue, 2 Feb 2016 17:56:59 +0000 (17:56 +0000)]
Add -C link-dead-code option r=alexcrichton
Turning gc-sections off improves code coverage based for tools which
use DWARF debugging information (like kcov). Otherwise dead code is
stripped and kcov returns a coverage percentage that doesn't reflect
reality.
bors [Thu, 11 Feb 2016 04:27:13 +0000 (04:27 +0000)]
Auto merge of #31461 - jseyfried:remove_import_resolutions, r=nrc
This PR adds to `NameBinding` so it can more fully represent bindings from imports as well from items, refactors away `Target`, generalizes `ImportResolution` to a simpler type `NameResolution`, and uses a single `NameResolution`-valued map in place the existing maps `children` and `import_resolutions` (of `NameBinding`s and `ImportResolution`s, respectively), simplifying duplicate checking and name resolution.
It also unifies the `resolve_name_in_module` in `lib.rs` with its namesake in `resolve_imports.rs`, clarifying and improving the core logic (fixes #31403 and fixes #31404) while maintaining clear future-comparability with shadowable globs (i.e., never reporting that a resolution is a `Success` or is `Failing` unless this would also be knowable with shadowable globs).
Since it fixes #31403, this is technically a [breaking-change], but it is exceedingly unlikely to cause breakage in practice. The following is an example of code that would break:
```rust
mod foo {
pub mod bar {} // This defines bar in the type namespace
pub use alpha::bar; // This defines bar in the value namespace
// This should define baz in both namespaces, but it only defines baz in the type namespace.
pub use self::bar as baz;
pub fn baz() {} // This should collide with baz, but now it does not.
}
pub fn f() {}
mod alpha {
pub use self::f as bar; // Changing this to `pub fn bar() {}` causes the collision right now.
pub use super::*;
}
```
bors [Thu, 11 Feb 2016 01:54:15 +0000 (01:54 +0000)]
Auto merge of #31479 - kamalmarhubi:fmt-pointer-unsized, r=alexcrichton
This allows printing pointers to unsized types with the {:p} formatting
directive. The following impls are extended to unsized types:
- impl<'a, T: ?Sized> Pointer for &'a T
- impl<'a, T: ?Sized> Pointer for &'a mut T
- impl<T: ?Sized> Pointer for *const T
- impl<T: ?Sized> Pointer for *mut T
- impl<T: ?Sized> fmt::Pointer for Box<T>
- impl<T: ?Sized> fmt::Pointer for Rc<T>
- impl<T: ?Sized> fmt::Pointer for Arc<T>
bors [Wed, 10 Feb 2016 22:51:43 +0000 (22:51 +0000)]
Auto merge of #31409 - alexcrichton:command-exec, r=aturon
These commits are an implementation of https://github.com/rust-lang/rfcs/pull/1359 which is tracked via https://github.com/rust-lang/rust/issues/31398. The `before_exec` implementation fit easily with the current process spawning framework we have, but unfortunately the `exec` implementation required a bit of a larger refactoring. The stdio handles were all largely managed as implementation details of `std::process` and the `exec` function lived in `std::sys`, so the two didn't have access to one another.
I took this as a sign that a deeper refactoring was necessary, and I personally feel that the end result is cleaner for both Windows and Unix. The commits should be separated nicely for reviewing (or all at once if you're feeling ambitious), but the changes made here were:
* The process spawning on Unix was refactored in to a pre-exec and post-exec function. The post-exec function isn't allowed to do any allocations of any form, and management of transmitting errors back to the parent is managed by the pre-exec function (as it's the one that actually forks).
* Some management of the exit status was pushed into platform-specific modules. On Unix we must cache the return value of `wait` as the pid is consumed after we wait on it, but on Windows we can just keep querying the system because the handle stays valid.
* The `Stdio::None` variant was renamed to `Stdio::Null` to better reflect what it's doing.
* The global lock on `CreateProcess` is now correctly positioned to avoid unintended inheritance of pipe handles that other threads are sending to their child processes. After a more careful reading of the article referenced the race is not in `CreateProcess` itself, but rather the property that handles are unintentionally shared.
* All stdio management now happens in platform-specific modules. This provides a cleaner implementation/interpretation for `FromFraw{Fd,Handle}` for each platform as well as a cleaner transition from a configuration to what-to-do once we actually need to do the spawn.
With these refactorings in place, implementing `before_exec` and `exec` ended up both being pretty trivial! (each in their own commit)
Björn Steinbrink [Fri, 29 Jan 2016 10:11:24 +0000 (11:11 +0100)]
Workaround LLVM optimizer bug by not marking &mut pointers as noalias
LLVM's memory dependence analysis doesn't properly account for calls
that could unwind and thus effectively act as a branching point. This
can lead to stores that are only visible when the call unwinds being
removed, possibly leading to calls to drop() functions with b0rked
memory contents.
As there is no fix for this in LLVM yet and we want to keep
compatibility to current LLVM versions anyways, we have to workaround
this bug by omitting the noalias attribute on &mut function arguments.
Benchmarks suggest that the performance loss by this change is very
small.
Thanks to @RalfJung for pushing me towards not removing too many
noalias annotations and @alexcrichton for helping out with the test for
this bug.
bors [Wed, 10 Feb 2016 20:55:53 +0000 (20:55 +0000)]
Auto merge of #31455 - tmiasko:expected-tokens, r=alexcrichton
Previously when breaking tokens into smaller pieces, the replace_token
function have been used. It replaced current token and updated span
information, but it did not clear the list of expected tokens, neither
did it update remaining info about last token. This could lead to
incorrect error message, like one described in the issue #24780:
Alex Crichton [Thu, 4 Feb 2016 19:16:32 +0000 (11:16 -0800)]
std: Implement CommandExt::exec
This commit implements the `exec` function proposed in [RFC 1359][rfc] which is
a function on the `CommandExt` trait to execute all parts of a `Command::spawn`
without the `fork` on Unix. More details on the function itself can be found in
the comments in the commit.
Alex Crichton [Thu, 4 Feb 2016 19:10:37 +0000 (11:10 -0800)]
std: Push process stdio setup in std::sys
Most of this is platform-specific anyway, and we generally have to jump through
fewer hoops to do the equivalent operation on Windows. One benefit for Windows
today is that this new structure avoids an extra `DuplicateHandle` when creating
pipes. For Unix, however, the behavior should be the same.
Note that this is just a pure refactoring, no functionality was added or
removed.
Alex Crichton [Thu, 4 Feb 2016 17:59:47 +0000 (09:59 -0800)]
std: Lift out Windows' CreateProcess lock a bit
The function `CreateProcess` is not itself unsafe to call from many threads, the
article in question is pointing out that handles can be inherited by unintended
child processes. This is basically the same race as the standard Unix
open-then-set-cloexec race.
Since the intention of the lock is to protect children from inheriting
unintended handles, the lock is now lifted out to before the creation of the
child I/O handles (which will all be inheritable). This will ensure that we only
have one process in Rust at least creating inheritable handles at a time,
preventing unintended inheritance to children.
Alex Crichton [Thu, 4 Feb 2016 02:09:35 +0000 (18:09 -0800)]
std: Push Child's exit status to sys::process
On Unix we have to be careful to not call `waitpid` twice, but we don't have to
be careful on Windows due to the way process handles work there. As a result the
cached `Option<ExitStatus>` is only necessary on Unix, and it's also just an
implementation detail of the Unix module.
At the same time. also update some code in `kill` on Unix to avoid a wonky
waitpid with WNOHANG. This was added in 0e190b9a to solve #13124, but the
`signal(0)` method is not supported any more so there's no need to for this
workaround. I believe that this is no longer necessary as it's not really doing
anything.
Alex Crichton [Thu, 4 Feb 2016 00:55:59 +0000 (16:55 -0800)]
std: Implement CommandExt::before_exec
This is a Unix-specific function which adds the ability to register a closure to
run pre-exec to configure the child process as required (note that these
closures are run post-fork).
Alex Crichton [Sat, 31 Oct 2015 18:09:43 +0000 (11:09 -0700)]
std: Refactor process spawning on Unix
* Build up the argp/envp pointers while the `Command` is being constructed
rather than only when `spawn` is called. This will allow better sharing of
code between fork/exec paths.
* Rename `child_after_fork` to `exec` and have it only perform the exec half of
the spawning. This also means the return type has changed to `io::Error`
rather than `!` to represent errors that happen.
bors [Wed, 10 Feb 2016 17:23:40 +0000 (17:23 +0000)]
Auto merge of #30686 - wesleywiser:rustdoc_display_since, r=steveklabnik
Here's some screenshots after this change:
![screen shot 2016-01-03 at 11 38 30 am](https://cloud.githubusercontent.com/assets/831192/12079661/23da4e38-b20f-11e5-8c84-ba51d7a59c3f.png)
![screen shot 2016-01-03 at 11 40 39 am](https://cloud.githubusercontent.com/assets/831192/12079663/23e00012-b20f-11e5-9f01-408cc8d43687.png)
![screen shot 2016-01-03 at 11 42 17 am](https://cloud.githubusercontent.com/assets/831192/12079662/23dfe6c2-b20f-11e5-9998-53abc643e2ef.png)
I tried to click through the `std` docs and make sure everything that can have stability attributes has it rendered but I'm probably missing some. I'd also appreciate any feedback on the css changes. I had difficulty getting the `since` labels aligning correctly for enum variants. If anyone has a better idea for that, I'd be glad to implement it.
bors [Wed, 10 Feb 2016 14:24:41 +0000 (14:24 +0000)]
Auto merge of #31499 - kamalmarhubi:cfg-flag-invalid-cfgs, r=brson
A spec like `#[cfg(foo(bar))]` is not allowed as an attribute. This
makes the same spec be rejected by the compiler if passed in as a
`--cfg` argument.
bors [Wed, 10 Feb 2016 10:04:46 +0000 (10:04 +0000)]
Auto merge of #31420 - bluss:deque-equality, r=Gankro
collections: Use slice parts in PartialEq for VecDeque
This improves == for VecDeque by using the slice representation.
This will also improve further if codegen for slice comparison improves.
Benchmark run of 1000 u64 elements, comparing for equality (all equal).
Cpu time to compare the vecdeques is reduced to less than 50% of what it
was before.
bors [Wed, 10 Feb 2016 08:03:06 +0000 (08:03 +0000)]
Auto merge of #31494 - alexcrichton:ar-gnu-by-default, r=brson
The compiler currently vendors its own version of "llvm-ar" (not literally the
binary but rather the library support) and uses it for all major targets by
default (e.g. everything defined in `src/librustc_back/target`). All custom
target specs, however, still search for an `ar` tool by default. This commit
changes this default behavior to using the internally bundled llvm-ar with the
GNU format.
Currently all targets use the GNU format except for OSX which uses the BSD
format (surely makes sense, right?), and custom targets can change the format
via the `archive-format` key in custom target specs.
I suspect that we can outright remove support for invoking an external `ar`
utility, but I figure for now there may be some crazy target relying on that so
we should leave support in for now.
bors [Wed, 10 Feb 2016 01:05:42 +0000 (01:05 +0000)]
Auto merge of #31438 - aturon:stab-ip-addr, r=alexcrichton
After [considerable pushback](https://github.com/rust-lang/rfcs/issues/1451), it's clear that there is a community consensus around providing `IpAddr` in the standard library, together with other APIs using it.
This commit reverts from deprecated status directly to stable. The deprecation landed in 1.6, which has already been released, so the stabilization is marked for 1.7 (currently in beta; will require a backport).