bors [Mon, 28 Oct 2013 01:41:25 +0000 (18:41 -0700)]
auto merge of #10098 : alexcrichton/rust/attempts, r=alexcrichton
It was pretty much a miracle that these tests were ever passing. They would
never have passed in the single threaded case because only one sigint in the
tests is ever generated, but when run in parallel two sigints will be generated.
Alex Crichton [Sun, 27 Oct 2013 17:58:32 +0000 (10:58 -0700)]
Ignore a test which never completes on windows
I'm not entirely sure why this is happening, but the server task is never seeing
the second send of the client task, and this test will very reliably fail to
complete on windows.
Alex Crichton [Sun, 27 Oct 2013 06:31:14 +0000 (23:31 -0700)]
Fix a typo in a rt::io::signal test
It was pretty much a miracle that these tests were ever passing. They would
never have passed in the single threaded case because only one sigint in the
tests is ever generated, but when run in parallel two sigints will be generated.
bors [Sat, 26 Oct 2013 08:11:11 +0000 (01:11 -0700)]
auto merge of #10048 : ibukanov/rust/mkdir-fix, r=brson
The patch replaces mkdir -p dir with (umask 022 && mkdir -p dir). Sadly mkdir -m 755 -p dir does not work if it creates parent directories as those parents will have umask permissions, not the one passed with -m option.
bors [Sat, 26 Oct 2013 07:06:09 +0000 (00:06 -0700)]
auto merge of #10070 : alexcrichton/rust/fewer-missiles, r=brson
This optimizes the `home_for_io` code path by requiring fewer scheduler
operations in some situtations.
When moving to your home scheduler, this no longer forces a context switch if
you're already on the home scheduler. Instead, the homing code now simply pins
you to your current scheduler (making it so you can't be stolen away). If you're
not on your home scheduler, then we context switch away, sending you to your
home scheduler.
When the I/O operation is done, then we also no longer forcibly trigger a
context switch. Instead, the action is cased on whether the task is homed or
not. If a task does not have a home, then the task is re-flagged as not having a
home and no context switch is performed. If a task is homed to the current
scheduler, then we don't do anything, and if the task is homed to a foreign
scheduler, then it's sent along its merry way.
I verified that there are about a third as many `write` syscalls done in print
operations now. Libuv uses write to implement async handles, and the homing
before and after each I/O operation was triggering a write on these async
handles. Additionally, using the terrible benchmark of printing 10k times in a
loop, this drives the runtime from 0.6s down to 0.3s (yay!).
Alex Crichton [Fri, 25 Oct 2013 18:11:30 +0000 (11:11 -0700)]
Fire fewer homing missiles
This optimizes the `home_for_io` code path by requiring fewer scheduler
operations in some situtations.
When moving to your home scheduler, this no longer forces a context switch if
you're already on the home scheduler. Instead, the homing code now simply pins
you to your current scheduler (making it so you can't be stolen away). If you're
not on your home scheduler, then we context switch away, sending you to your
home scheduler.
When the I/O operation is done, then we also no longer forcibly trigger a
context switch. Instead, the action is cased on whether the task is homed or
not. If a task does not have a home, then the task is re-flagged as not having a
home and no context switch is performed. If a task is homed to the current
scheduler, then we don't do anything, and if the task is homed to a foreign
scheduler, then it's sent along its merry way.
I verified that there are about a third as many `write` syscalls done in print
operations now. Libuv uses write to implement async handles, and the homing
before and after each I/O operation was triggering a write on these async
handles. Additionally, using the terrible benchmark of printing 10k times in a
loop, this drives the runtime from 0.6s down to 0.3s (yay!).
bors [Fri, 25 Oct 2013 17:36:09 +0000 (10:36 -0700)]
auto merge of #10060 : alexcrichton/rust/cached-stdout, r=brson
Almost all languages provide some form of buffering of the stdout stream, and
this commit adds this feature for rust. A handle to stdout is lazily initialized
in the Task structure as a buffered owned Writer trait object. The buffer
behavior depends on where stdout is directed to. Like C, this line-buffers the
stream when the output goes to a terminal (flushes on newlines), and also like C
this uses a fixed-size buffer when output is not directed at a terminal.
We may decide the fixed-size buffering is overkill, but it certainly does reduce
write syscall counts when piping output elsewhere. This is a *huge* benefit to
any code using logging macros or the printing macros. Formatting emits calls to
`write` very frequently, and to have each of them backed by a write syscall was
very expensive.
In a local benchmark of printing 10000 lines of "what" to stdout, I got the
following timings:
when | terminal | redirected
----------|---------------|--------
before | 0.575s | 0.525s
after | 0.197s | 0.013s
C | 0.019s | 0.004s
I can also confirm that we're buffering the output appropriately in both
situtations. We're still far slower than C, but I believe much of that has to do
with the "homing" that all tasks due, we're still performing an order of
magnitude more write syscalls than C does.
Alex Crichton [Fri, 25 Oct 2013 00:30:36 +0000 (17:30 -0700)]
Cache and buffer stdout per-task for printing
Almost all languages provide some form of buffering of the stdout stream, and
this commit adds this feature for rust. A handle to stdout is lazily initialized
in the Task structure as a buffered owned Writer trait object. The buffer
behavior depends on where stdout is directed to. Like C, this line-buffers the
stream when the output goes to a terminal (flushes on newlines), and also like C
this uses a fixed-size buffer when output is not directed at a terminal.
We may decide the fixed-size buffering is overkill, but it certainly does reduce
write syscall counts when piping output elsewhere. This is a *huge* benefit to
any code using logging macros or the printing macros. Formatting emits calls to
`write` very frequently, and to have each of them backed by a write syscall was
very expensive.
In a local benchmark of printing 10000 lines of "what" to stdout, I got the
following timings:
when | terminal | redirected
----------------------------------
before | 0.575s | 0.525s
after | 0.197s | 0.013s
C | 0.019s | 0.004s
I can also confirm that we're buffering the output appropriately in both
situtations. We're still far slower than C, but I believe much of that has to do
with the "homing" that all tasks due, we're still performing an order of
magnitude more write syscalls than C does.
bors [Fri, 25 Oct 2013 07:46:11 +0000 (00:46 -0700)]
auto merge of #10054 : alexcrichton/rust/basic-event-loop, r=brson
This is more progress towards #9128 and all its related tree of issues. This implements a new `BasicLoop` on top of pthreads synchronization primitives (wrapped in `LittleLock`). This also removes the wonky `callback_ms` function from the interface of the event loop.
After #9901 is taking forever to land, I'm going to try to do all this runtime work in much smaller chunks than before. Right now this will not work unless #9901 lands first, but I'm close to landing it (hopefully), and I wanted to go ahead and get this reviewed before throwing it at bors later on down the road.
This "pausible idle callback" is also a bit of a weird idea, but it wasn't as difficult to implement as callback_ms so I'm more semi-ok with it.
Alex Crichton [Tue, 22 Oct 2013 22:00:37 +0000 (15:00 -0700)]
Implement a basic event loop built on LittleLock
It's not guaranteed that there will always be an event loop to run, and this
implementation will serve as an incredibly basic one which does not provide any
I/O, but allows the scheduler to still run.
Alex Crichton [Thu, 24 Oct 2013 18:30:35 +0000 (11:30 -0700)]
Remove the 'callback_ms' function from EventLoop
This is a peculiar function to require event loops to implement, and it's only
used in one spot during tests right now. Instead, a possibly more robust apis
for timers should be used rather than requiring all event loops to implement a
curious-looking function.
Alex Crichton [Tue, 22 Oct 2013 21:59:21 +0000 (14:59 -0700)]
Fix a bug with the scheduler and destructor order
The PausibleIdleCallback must have some handle into the event loop, and because
struct destructors are run in order of top-to-bottom in order of fields, this
meant that the event loop was getting destroyed before the idle callback was
getting destroyed.
I can't confirm that this fixes a problem in how we use libuv, but it does
semantically fix a problem for usage with other event loops.
bors [Thu, 24 Oct 2013 21:26:15 +0000 (14:26 -0700)]
auto merge of #9901 : alexcrichton/rust/unix-sockets, r=brson
Large topics:
* Implemented `rt::io::net::unix`. We've got an implementation backed by "named pipes" for windows for free from libuv, so I'm not sure if these should be `cfg(unix)` or whether they'd be better placed in `rt::io::pipe` (which is currently kinda useless), or to leave in `unix`. Regardless, we probably shouldn't deny windows of functionality which it certainly has.
* Fully implemented `net::addrinfo`, or at least fully implemented in the sense of making the best attempt to wrap libuv's `getaddrinfo` api
* Moved standard I/O to a libuv TTY instead of just a plain old file descriptor. I found that this interacted better when closing stdin, and it has the added bonus of getting things like terminal dimentions (someone should make a progress bar now!)
* Migrate to `~Trait` instead of a typedef'd object where possible. There are only two more types which are blocked on this, and those are traits which have a method which takes by-value self (there's an open issue on this)
* Drop `rt::io::support::PathLike` in favor of just `ToCStr`. We recently had a lot of Path work done, but it still wasn't getting passed down to libuv (there was an intermediate string conversion), and this allows true paths to work all the way down to libuv (and anything else that can become a C string).
* Removes `extra::fileinput` and `extra::io_util`
Closes #9895
Closes #9975
Closes #8330
Closes #6850 (ported lots of libraries away from std::io)
cc #4248 (implemented unix/dns)
cc #9128 (made everything truly trait objects)
Alex Crichton [Tue, 22 Oct 2013 15:41:05 +0000 (08:41 -0700)]
Fixing some tests, adding some pipes
This adds constructors to pipe streams in the new runtime to take ownership of
file descriptors, and also fixes a few tests relating to the std::run changes
(new errors are raised on io_error and one test is xfail'd).
Alex Crichton [Fri, 18 Oct 2013 21:01:22 +0000 (14:01 -0700)]
Move stdin to using libuv's pipes instead of a tty
I was seeing a lot of weird behavior with stdin behaving as a tty, and it
doesn't really quite make sense, so instead this moves to using libuv's pipes
instead (which make more sense for stdin specifically).
This prevents piping input to rustc hanging forever.
Alex Crichton [Fri, 18 Oct 2013 18:52:23 +0000 (11:52 -0700)]
Remove io::read_error
The general idea is to remove conditions completely from I/O, so in the meantime
remove the read_error condition to mean the same thing as the io_error condition.
Alex Crichton [Fri, 18 Oct 2013 01:51:32 +0000 (18:51 -0700)]
Stop logging task failure to task loggers
The isn't an ideal patch, and the comment why is in the code. Basically uvio
uses task::unkillable which touches the kill flag for a task, and if the task is
failing due to mismangement of the kill flag, then there will be serious
problems when the task tries to print that it's failing.
Alex Crichton [Fri, 18 Oct 2013 00:04:51 +0000 (17:04 -0700)]
Move as much I/O as possible off of native::io
When uv's TTY I/O is used for the stdio streams, the file descriptors are put
into a non-blocking mode. This means that other concurrent writes to the same
stream can fail with EAGAIN or EWOULDBLOCK. By all I/O to event-loop I/O, we
avoid this error.
There is one location which cannot move, which is the runtime's dumb_println
function. This was implemented to handle the EAGAIN and EWOULDBLOCK errors and
simply retry again and again.
Alex Crichton [Thu, 17 Oct 2013 00:05:28 +0000 (17:05 -0700)]
Remove IoFactoryObject for ~IoFactory
This involved changing a fair amount of code, rooted in how we access the local
IoFactory instance. I added a helper method to the rtio module to access the
optional local IoFactory. This is different than before in which it was assumed
that a local IoFactory was *always* present. Now, a separate io_error is raised
when an IoFactory is not present, yet I/O is requested.
Alex Crichton [Wed, 16 Oct 2013 23:48:30 +0000 (16:48 -0700)]
Remove rt::io::support
This removes the PathLike trait associated with this "support module". This is
yet another "container of bytes" trait, so I didn't want to duplicate what
already exists throughout libstd. In actuality, we're going to pass of C strings
to the libuv APIs, so instead the arguments are now bound with the 'ToCStr'
trait instead.
Additionally, a layer of complexity was removed by immediately converting these
type-generic parameters into CStrings to get handed off to libuv apis.
Alex Crichton [Wed, 16 Oct 2013 18:47:12 +0000 (11:47 -0700)]
Move rt::io::stdio from FileStream to a TTY
We get a little more functionality from libuv for these kinds of streams (things
like terminal dimentions), and it also appears to more gracefully handle the
stream being a window. Beforehand, if you used stdio and hit CTRL+d on a
process, libuv would continually return 0-length successful reads instead of
interpreting that the stream was closed.
I was hoping to be able to write tests for this, but currently the testing
infrastructure doesn't allow tests with a stdin and a stdout, but this has been
manually tested! (not that it means much)
bors [Wed, 23 Oct 2013 15:31:21 +0000 (08:31 -0700)]
auto merge of #9810 : huonw/rust/rand3, r=alexcrichton
- Adds the `Sample` and `IndependentSample` traits for generating numbers where there are parameters (e.g. a list of elements to draw from, or the mean/variance of a normal distribution). The former takes `&mut self` and the latter takes `&self` (this is the only difference).
- Adds proper `Normal` and `Exp`-onential distributions
- Adds `Range` which generates `[lo, hi)` generically & properly (via a new trait) replacing the incorrect behaviour of `Rng.gen_integer_range` (this has become `Rng.gen_range` for convenience, it's far more efficient to use `Range` itself)
- Move the `Weighted` struct from `std::rand` to `std::rand::distributions` & improve it
- optimisations and docs
bors [Wed, 23 Oct 2013 05:51:10 +0000 (22:51 -0700)]
auto merge of #10015 : huonw/rust/minor-fixes, r=alexcrichton
- Use ["nothing up my sleeve numbers"](http://en.wikipedia.org/wiki/Nothing_up_my_sleeve_number) for the ISAAC tests.
- Replace the default implementation of `Rng.fill_bytes` with something that doesn't try to do bad things with `transmute` and vectors just for the sake of a little speed.
- Replace the transmutes used to seed the ISAAC RNGs with calls into `vec::raw`.
bors [Wed, 23 Oct 2013 03:56:05 +0000 (20:56 -0700)]
auto merge of #9654 : catamorphism/rust/rustpkg-c-dependencies, r=brson
r? @brson api::install_pkg now accepts an argument that's a list of
(kind, path) dependency pairs. This allows custom package scripts to
declare C dependencies, as is demonstrated in
rustpkg::tests::test_c_dependency_ok.
Tim Chevalier [Tue, 1 Oct 2013 05:39:50 +0000 (22:39 -0700)]
rustpkg: Support arbitrary dependencies in the install API
api::install_pkg now accepts an argument that's a list of
(kind, path) dependency pairs. This allows custom package scripts to
declare C dependencies, as is demonstrated in
rustpkg::tests::test_c_dependency_ok.
Huon Wilson [Tue, 22 Oct 2013 12:11:17 +0000 (23:11 +1100)]
std::rand: seed ISAAC with no transmutes.
Slice transmutes are now (and, really, always were) dangerous, so we
avoid them and do the (only?) non-(undefined behaviour in C) pointer
cast: casting to *u8.