1 % Rust Foreign Function Interface Tutorial
5 This tutorial will use the [snappy](https://code.google.com/p/snappy/)
6 compression/decompression library as an introduction to writing bindings for
7 foreign code. Rust is currently unable to call directly into a C++ library, but
8 snappy includes a C interface (documented in
9 [`snappy-c.h`](https://code.google.com/p/snappy/source/browse/trunk/snappy-c.h)).
11 The following is a minimal example of calling a foreign function which will compile if snappy is
15 use std::libc::size_t;
17 #[link_args = "-lsnappy"]
19 fn snappy_max_compressed_length(source_length: size_t) -> size_t;
22 #[fixed_stack_segment]
24 let x = unsafe { snappy_max_compressed_length(100) };
25 println(fmt!("max compressed length of a 100 byte buffer: %?", x));
29 The `extern` block is a list of function signatures in a foreign library, in this case with the
30 platform's C ABI. The `#[link_args]` attribute is used to instruct the linker to link against the
31 snappy library so the symbols are resolved.
33 Foreign functions are assumed to be unsafe so calls to them need to be wrapped with `unsafe {}` as a
34 promise to the compiler that everything contained within truly is safe. C libraries often expose
35 interfaces that aren't thread-safe, and almost any function that takes a pointer argument isn't
36 valid for all possible inputs since the pointer could be dangling, and raw pointers fall outside of
37 Rust's safe memory model.
39 Finally, the `#[fixed_stack_segment]` annotation that appears on
40 `main()` instructs the Rust compiler that when `main()` executes, it
41 should request a "very large" stack segment. More details on
42 stack management can be found in the following sections.
44 When declaring the argument types to a foreign function, the Rust compiler will not check if the
45 declaration is correct, so specifying it correctly is part of keeping the binding correct at
48 The `extern` block can be extended to cover the entire snappy API:
51 use std::libc::{c_int, size_t};
53 #[link_args = "-lsnappy"]
55 fn snappy_compress(input: *u8,
58 compressed_length: *mut size_t) -> c_int;
59 fn snappy_uncompress(compressed: *u8,
60 compressed_length: size_t,
61 uncompressed: *mut u8,
62 uncompressed_length: *mut size_t) -> c_int;
63 fn snappy_max_compressed_length(source_length: size_t) -> size_t;
64 fn snappy_uncompressed_length(compressed: *u8,
65 compressed_length: size_t,
66 result: *mut size_t) -> c_int;
67 fn snappy_validate_compressed_buffer(compressed: *u8,
68 compressed_length: size_t) -> c_int;
72 # Creating a safe interface
74 The raw C API needs to be wrapped to provide memory safety and make use of higher-level concepts
75 like vectors. A library can choose to expose only the safe, high-level interface and hide the unsafe
78 Wrapping the functions which expect buffers involves using the `vec::raw` module to manipulate Rust
79 vectors as pointers to memory. Rust's vectors are guaranteed to be a contiguous block of memory. The
80 length is number of elements currently contained, and the capacity is the total size in elements of
81 the allocated memory. The length is less than or equal to the capacity.
84 #[fixed_stack_segment]
86 pub fn validate_compressed_buffer(src: &[u8]) -> bool {
88 snappy_validate_compressed_buffer(vec::raw::to_ptr(src), src.len() as size_t) == 0
93 The `validate_compressed_buffer` wrapper above makes use of an `unsafe` block, but it makes the
94 guarantee that calling it is safe for all inputs by leaving off `unsafe` from the function
97 The `validate_compressed_buffer` wrapper is also annotated with two
98 attributes `#[fixed_stack_segment]` and `#[inline(never)]`. The
99 purpose of these attributes is to guarantee that there will be
100 sufficient stack for the C function to execute. This is necessary
101 because Rust, unlike C, does not assume that the stack is allocated in
102 one continuous chunk. Instead, we rely on a *segmented stack* scheme,
103 in which the stack grows and shrinks as necessary. C code, however,
104 expects one large stack, and so callers of C functions must request a
105 large stack segment to ensure that the C routine will not run off the
108 The compiler includes a lint mode that will report an error if you
109 call a C function without a `#[fixed_stack_segment]` attribute. More
110 details on the lint mode are given in a later section.
112 You may be wondering why we include a `#[inline(never)]` directive.
113 This directive informs the compiler never to inline this function.
114 While not strictly necessary, it is usually a good idea to use an
115 `#[inline(never)]` directive in concert with `#[fixed_stack_segment]`.
116 The reason is that if a fn annotated with `fixed_stack_segment` is
117 inlined, then its caller also inherits the `fixed_stack_segment`
118 annotation. This means that rather than requesting a large stack
119 segment only for the duration of the call into C, the large stack
120 segment would be used for the entire duration of the caller. This is
121 not necessarily *bad* -- it can for example be more efficient,
122 particularly if `validate_compressed_buffer()` is called multiple
123 times in a row -- but it does work against the purpose of the
124 segmented stack scheme, which is to keep stacks small and thus
125 conserve address space.
127 The `snappy_compress` and `snappy_uncompress` functions are more complex, since a buffer has to be
128 allocated to hold the output too.
130 The `snappy_max_compressed_length` function can be used to allocate a vector with the maximum
131 required capacity to hold the compressed output. The vector can then be passed to the
132 `snappy_compress` function as an output parameter. An output parameter is also passed to retrieve
133 the true length after compression for setting the length.
136 pub fn compress(src: &[u8]) -> ~[u8] {
137 #[fixed_stack_segment]; #[inline(never)];
140 let srclen = src.len() as size_t;
141 let psrc = vec::raw::to_ptr(src);
143 let mut dstlen = snappy_max_compressed_length(srclen);
144 let mut dst = vec::with_capacity(dstlen as uint);
145 let pdst = vec::raw::to_mut_ptr(dst);
147 snappy_compress(psrc, srclen, pdst, &mut dstlen);
148 vec::raw::set_len(&mut dst, dstlen as uint);
154 Decompression is similar, because snappy stores the uncompressed size as part of the compression
155 format and `snappy_uncompressed_length` will retrieve the exact buffer size required.
158 pub fn uncompress(src: &[u8]) -> Option<~[u8]> {
159 #[fixed_stack_segment]; #[inline(never)];
162 let srclen = src.len() as size_t;
163 let psrc = vec::raw::to_ptr(src);
165 let mut dstlen: size_t = 0;
166 snappy_uncompressed_length(psrc, srclen, &mut dstlen);
168 let mut dst = vec::with_capacity(dstlen as uint);
169 let pdst = vec::raw::to_mut_ptr(dst);
171 if snappy_uncompress(psrc, srclen, pdst, &mut dstlen) == 0 {
172 vec::raw::set_len(&mut dst, dstlen as uint);
175 None // SNAPPY_INVALID_INPUT
181 For reference, the examples used here are also available as an [library on
182 GitHub](https://github.com/thestinger/rust-snappy).
186 Sometimes writing Rust wrappers can be quite tedious. For example, if
187 function does not take any pointer arguments, often there is no need
188 for translating types. In such cases, it is usually still a good idea
189 to have a Rust wrapper so as to manage the segmented stacks, but you
190 can take advantage of the (standard) `externfn!` macro to remove some
193 In the initial section, we showed an extern block that added a call
194 to a specific snappy API:
197 use std::libc::size_t;
199 #[link_args = "-lsnappy"]
201 fn snappy_max_compressed_length(source_length: size_t) -> size_t;
204 #[fixed_stack_segment]
206 let x = unsafe { snappy_max_compressed_length(100) };
207 println(fmt!("max compressed length of a 100 byte buffer: %?", x));
211 To avoid the need to create a wrapper fn for `snappy_max_compressed_length()`,
212 and also to avoid the need to think about `#[fixed_stack_segment]`, we
213 could simply use the `externfn!` macro instead, as shown here:
216 use std::libc::size_t;
218 externfn!(#[link_args = "-lsnappy"]
219 fn snappy_max_compressed_length(source_length: size_t) -> size_t)
222 let x = unsafe { snappy_max_compressed_length(100) };
223 println(fmt!("max compressed length of a 100 byte buffer: %?", x));
227 As you can see from the example, `externfn!` replaces the extern block
228 entirely. After macro expansion, it will create something like this:
231 use std::libc::size_t;
233 // Automatically generated by
234 // externfn!(#[link_args = "-lsnappy"]
235 // fn snappy_max_compressed_length(source_length: size_t) -> size_t)
236 unsafe fn snappy_max_compressed_length(source_length: size_t) -> size_t {
237 #[fixed_stack_segment]; #[inline(never)];
238 return snappy_max_compressed_length(source_length);
240 #[link_args = "-lsnappy"]
242 fn snappy_max_compressed_length(source_length: size_t) -> size_t;
247 let x = unsafe { snappy_max_compressed_length(100) };
248 println(fmt!("max compressed length of a 100 byte buffer: %?", x));
252 # Segmented stacks and the linter
254 By default, whenever you invoke a non-Rust fn, the `cstack` lint will
255 check that one of the following conditions holds:
257 1. The call occurs inside of a fn that has been annotated with
258 `#[fixed_stack_segment]`;
259 2. The call occurs inside of an `extern fn`;
260 3. The call occurs within a stack closure created by some other
263 All of these conditions ensure that you are running on a large stack
264 segmented. However, they are sometimes too strict. If your application
265 will be making many calls into C, it is often beneficial to promote
266 the `#[fixed_stack_segment]` attribute higher up the call chain. For
267 example, the Rust compiler actually labels main itself as requiring a
268 `#[fixed_stack_segment]`. In such cases, the linter is just an
269 annoyance, because all C calls that occur from within the Rust
270 compiler are made on a large stack. Another situation where this
271 frequently occurs is on a 64-bit architecture, where large stacks are
272 the default. In cases, you can disable the linter by including a
273 `#[allow(cstack)]` directive somewhere, which permits violations of
274 the "cstack" rules given above (you can also use `#[warn(cstack)]` to
275 convert the errors into warnings, if you prefer).
279 Foreign libraries often hand off ownership of resources to the calling code,
280 which should be wrapped in a destructor to provide safety and guarantee their
283 A type with the same functionality as owned boxes can be implemented by
284 wrapping `malloc` and `free`:
288 use std::libc::{c_void, size_t, malloc, free};
290 use std::unstable::intrinsics;
292 // a wrapper around the handle returned by the foreign code
293 pub struct Unique<T> {
297 impl<T: Send> Unique<T> {
298 pub fn new(value: T) -> Unique<T> {
299 #[fixed_stack_segment];
303 let ptr = malloc(std::sys::size_of::<T>() as size_t) as *mut T;
304 assert!(!ptr::is_null(ptr));
305 // `*ptr` is uninitialized, and `*ptr = value` would attempt to destroy it
306 intrinsics::move_val_init(&mut *ptr, value);
311 // the 'r lifetime results in the same semantics as `&*x` with ~T
312 pub fn borrow<'r>(&'r self) -> &'r T {
313 unsafe { cast::copy_lifetime(self, &*self.ptr) }
316 // the 'r lifetime results in the same semantics as `&mut *x` with ~T
317 pub fn borrow_mut<'r>(&'r mut self) -> &'r mut T {
318 unsafe { cast::copy_mut_lifetime(self, &mut *self.ptr) }
323 impl<T: Send> Drop for Unique<T> {
325 #[fixed_stack_segment];
329 let x = intrinsics::init(); // dummy value to swap in
330 // moving the object out is needed to call the destructor
331 ptr::replace_ptr(self.ptr, x);
332 free(self.ptr as *c_void)
337 // A comparison between the built-in ~ and this reimplementation
342 } // `x` is freed here
345 let mut y = Unique::new(5);
346 *y.borrow_mut() = 10;
347 } // `y` is freed here
353 In addition to the `#[link_args]` attribute for explicitly passing arguments to the linker, an
354 `extern mod` block will pass `-lmodname` to the linker by default unless it has a `#[nolink]`
359 Some operations, like dereferencing unsafe pointers or calling functions that have been marked
360 unsafe are only allowed inside unsafe blocks. Unsafe blocks isolate unsafety and are a promise to
361 the compiler that the unsafety does not leak out of the block.
363 Unsafe functions, on the other hand, advertise it to the world. An unsafe function is written like
367 unsafe fn kaboom(ptr: *int) -> int { *ptr }
370 This function can only be called from an `unsafe` block or another `unsafe` function.
372 # Accessing foreign globals
374 Foreign APIs often export a global variable which could do something like track
375 global state. In order to access these variables, you declare them in `extern`
376 blocks with the `static` keyword:
381 #[link_args = "-lreadline"]
383 static rl_readline_version: libc::c_int;
387 println(fmt!("You have readline version %d installed.",
388 rl_readline_version as int));
392 Alternatively, you may need to alter global state provided by a foreign
393 interface. To do this, statics can be declared with `mut` so rust can mutate
400 #[link_args = "-lreadline"]
402 static mut rl_prompt: *libc::c_char;
406 do "[my-awesome-shell] $".as_c_str |buf| {
407 unsafe { rl_prompt = buf; }
408 // get a line, process it
409 unsafe { rl_prompt = ptr::null(); }
414 # Foreign calling conventions
416 Most foreign code exposes a C ABI, and Rust uses the platform's C calling convention by default when
417 calling foreign functions. Some foreign functions, most notably the Windows API, use other calling
418 conventions. Rust provides the `abi` attribute as a way to hint to the compiler which calling
422 #[cfg(target_os = "win32")]
424 #[link_name = "kernel32"]
426 fn SetEnvironmentVariableA(n: *u8, v: *u8) -> int;
430 The `abi` attribute applies to a foreign module (it cannot be applied to a single function within a
431 module), and must be either `"cdecl"` or `"stdcall"`. The compiler may eventually support other
434 # Interoperability with foreign code
436 Rust guarantees that the layout of a `struct` is compatible with the platform's representation in C.
437 A `#[packed]` attribute is available, which will lay out the struct members without padding.
438 However, there are currently no guarantees about the layout of an `enum`.
440 Rust's owned and managed boxes use non-nullable pointers as handles which point to the contained
441 object. However, they should not be manually created because they are managed by internal
442 allocators. Borrowed pointers can safely be assumed to be non-nullable pointers directly to the
443 type. However, breaking the borrow checking or mutability rules is not guaranteed to be safe, so
444 prefer using raw pointers (`*`) if that's needed because the compiler can't make as many assumptions
447 Vectors and strings share the same basic memory layout, and utilities are available in the `vec` and
448 `str` modules for working with C APIs. However, strings are not terminated with `\0`. If you need a
449 NUL-terminated string for interoperability with C, you should use the `c_str::to_c_str` function.
451 The standard library includes type aliases and function definitions for the C standard library in
452 the `libc` module, and Rust links against `libc` and `libm` by default.