1 .HTML "Changes to the Programming Environment in the Fourth Release of Plan 9
4 Changes to the Programming Environment
8 Fourth Release of Plan 9
12 rob@plan9.bell-labs.com
16 The fourth release of Plan 9 includes changes at many levels of the system,
17 with repercussions in the libraries and program interfaces.
18 This document summarizes the changes and describes how
19 existing programs must be modified to run in the new release.
20 It is not exhaustive, of course; for further detail about any of the
21 topics refer to the manual pages, as always.
23 Programmers new to Plan 9 may find valuable tidbits here, but the
24 real audience for this paper is those with a need to update applications
25 and servers written in C for earlier releases of the Plan 9 operating system.
27 9P, NAMELEN, and strings
29 The underlying file service protocol for Plan 9, 9P, retains its basic form
30 but has had a number of adjustments to deal with longer file names and error strings,
31 new authentication mechanisms, and to make it more efficient at
32 evaluating file names.
33 The change to file names affects a number of system interfaces;
34 because file name elements are no longer of fixed size, they can
35 no longer be stored as arrays.
37 9P used to be a fixed-format protocol with
39 byte arrays representing file name elements.
40 Now, it is a variable-format protocol, as described in
42 in which strings are represented by a count followed by that many bytes.
45 would previously have occupied 28
47 bytes in the message; now it occupies 5: a two-byte count followed by the three bytes of
50 (And of course, a name could now be much longer.)
51 A similar format change has been made to
53 buffers: they are no longer
55 bytes long but instead have variable size prefixed by a two-byte count.
56 And in fact the entire 9P message syntax has changed: every message
57 now begins with a message length field that makes it trivial to break the
58 string into messages without parsing them, so
61 A new library entry point,
63 makes it easy for user-level servers to break the client data stream into 9P messages.
64 All servers should switch from using
71 This change to 9P affects the way strings are handled by the kernel and throughout
73 The consequences are primarily that fixed-size arrays have been replaced
74 by pointers and counts in a variety of system interfaces.
75 Most programs will need at least some adjustment to the new style.
78 is gone, except as a vestige in the authentication libraries, where it has been
85 All programs that mention
87 will need to be fixed.
89 The simplest place to see this change is in the
91 system call, which no longer assumes a buffer of length
93 but now requires a byte-count argument:
97 errstr(buf, sizeof buf);
99 The buffer can be any size you like.
100 For convenience, the kernel stores error strings internally as 256-byte arrays,
101 so if you like \(em but it's not required \(em you can use the defined constant
103 as a good buffer size.
106 (which had value 64),
108 is advisory, not mandatory, and is not part of the 9P specification.
110 With names, stat buffers, and directories, there isn't even an echo of a fixed-size array any more.
112 Directories and wait messages
114 With strings now variable-length, a number of system calls needed to change:
122 are all affected, as is
124 when applied to directories.
126 As far as directories are concerned, most programs don't use the system calls
127 directly anyway, since they operate on the machine-independent form, but
128 instead call the machine-dependent
134 These used to fill user-provided fixed-size buffers; now they return objects allocated
137 (which must therefore be freed after use).
142 d = dirstat(filename);
144 fprint(2, "can't stat %s: %r\en", filename);
150 A common new bug is to forget to free a
158 work pretty much as before, but changes to 9P make
159 it possible to exercise finer-grained control on what fields
162 are to be changed; see
168 Reading a directory works in a similar way to
172 allocating and filling in an array of
175 The return value is the number of elements of the array.
178 now include a pointer to a
180 to be filled in with the address of the allocated array:
185 while((n = dirread(fd, &d)) > 0){
191 A new library function,
195 but returns the entire directory in one call:
197 n = dirreadall(fd, &d)
202 If your program insists on using the underlying
204 system call or its relatives, or wants to operate directly on the
205 machine-independent format returned by
209 it will need to be modified.
210 Such programs are rare enough that we'll not discuss them here beyond referring to
214 Be aware, though, that it used to be possible to regard the buffer returned by
216 as a byte array that began with the zero-terminated
217 name of the file; this is no longer true.
218 With very rare exceptions, programs that call
220 would be better recast to use the
222 routines or, if their goal is just to test the existence of a file,
225 Similar changes have affected the
227 system call. In fact,
229 is no longer a system call but a library routine that calls the new
231 system call and returns a newly allocated machine-dependent
240 print("pid is %d; exit string %s\en", w->pid, w->msg);
245 may be empty but it will never be a nil pointer.
246 Again, don't forget to free the structure returned by
248 If all you need is the pid, you can call
250 which reports just the pid and doesn't return an allocated structure:
257 print("pid is %d\en", pid);
260 Quoted strings and tokenize
263 gives us a good opportunity to describe how the system copes with all this
265 Consider the text returned by the
267 system call, which includes a set of integers (pids and times) and a string (the exit status).
268 This information is formatted free-form; here is the statement in the kernel that
269 generates the message:
271 n = snprint(a, n, "%d %lud %lud %lud %q",
273 wq->w.time[TUser], wq->w.time[TSys], wq->w.time[TReal],
278 to produce a quoted-string representation of the exit status.
281 format is like %s but will wrap
283 single quotes around the string if it contains white space or is otherwise ambiguous.
286 can be used to parse data formatted this way: it splits white-space-separated
287 fields but understands the
292 library routine builds its
294 from the data returned by
301 char buf[512], *fld[5];
304 n = await(buf, sizeof buf-1);
308 if(tokenize(buf, fld, nelem(fld)) != nelem(fld)){
309 werrstr("couldn't parse wait message");
312 l = strlen(fld[4])+1;
313 w = malloc(sizeof(Waitmsg)+l);
316 w->pid = atoi(fld[0]);
317 w->time[0] = atoi(fld[1]);
318 w->time[1] = atoi(fld[2]);
319 w->time[2] = atoi(fld[3]);
320 w->msg = (char*)&w[1];
321 memmove(w->msg, fld[4], l);
326 This style of quoted-string and
328 is used all through the system now.
329 In particular, devices now
331 the messages written to their
333 files, which means that you can send messages that contain white space, by quoting them,
334 and that you no longer need to worry about whether or not the device accepts a newline.
335 In other words, you can say
337 echo message > /dev/xx/ctl
344 treats the newline character as white space and discards it.
346 While we're on the subject of quotes and strings, note that the implementation of
354 because it has no protection against buffer overflow.
359 to constrain the output.
362 format is cleverer than most in this regard:
363 if the string is too long to be represented in full,
365 is smart enough to produce a truncated but correctly quoted
366 string within the available space.
370 Although strings in 9P are now variable-length and not zero-terminated,
371 this has little direct effect in most of the system interfaces.
372 File and user names are still zero-terminated strings as always;
373 the kernel does the work of translating them as necessary for
375 And of course, they are now free to be as long as you might want;
376 the only hard limit is that their length must be represented in 16 bits.
378 One example where this matters is that the file system specification in the
380 system call can now be much longer.
383 that used the specification string in creative ways were limited by the
385 restriction; now they can use the string more freely.
387 now accepts a simple but less cryptic specification language for the window
392 % mount $wsys /mnt/wsys 'new -dx 250 -dy 250 -pid 1234'
394 In the old system, this sort of control was impossible through the
398 While we're on the subject of
400 note that with the new security architecture
403 9P has moved its authentication outside the protocol proper.
404 (For a full description of this change to 9P, see
408 .I "Security in Plan 9\f1.)
409 The most explicit effect of this change is that
411 now takes another argument,
413 a file descriptor for the
414 authentication file through which the authentication will be made.
415 For most user-level file servers, which do not require authentication, it is
416 sufficient to provide
421 if(mount(fd, -1, "/mnt/wsys", MREPL,
422 "new -dx 250 -dy 250 -pid 1234") < 0)
423 error("mount failed: %r");
425 To connect to servers that require authentication, use the new
427 system call or the reimplemented
429 (authenticated mount) library call.
432 handles both authenticating and non-authenticating servers, it is often
433 easiest just to replace calls to
443 The C library has been heavily reworked in places.
444 Besides the changes mentioned above, it
445 now has a much more complete set of routines for handling
447 strings (that is, zero-terminated arrays of 16-bit character values).
448 The most sweeping changes, however, are in the way formatted I/O is performed.
452 routine and all its relatives have been reimplemented to offer a number
455 Better buffer management, including the provision of an internal flush
456 routine, makes it unnecessary to provide large buffers.
459 uses a much smaller buffer now (reducing stack load) while simultaneously
460 removing the need to truncate the output string if it doesn't fit in the buffer.
462 Global variables have been eliminated so no locking is necessary.
464 The combination of (1) and (2) means that the standard implementation of
466 now works fine in threaded programs, and
472 prints into, and returns, storage allocated on demand by
475 It is now possible to print into a
477 string; for instance,
484 There is improved support for custom
485 print verbs and custom output routines such as error handlers.
490 can always be used instead.
491 However, the new routines
496 are often a better replacement.
497 The details are too long for exposition here;
499 explains the new interface and provides examples.
501 Two new format flags, space and comma, close somewhat the gap between
504 Despite these changes, most programs will be unaffected;
508 Don't forget, though, that
509 you should eliminate calls to
513 format when appropriate.
517 The discussion so far has been about changes at the source level.
518 Existing binaries will probably run without change in the new
519 environment, since the kernel provides backward-compatible
525 The only exceptions are programs that do either a
527 system call, because of the security changes and because
528 the file descriptor in
530 must point to a new 9P connection; or a
532 system call on a directory, since the returned data will
533 be in the new format.
534 A moment's reflection will discover that this means old
535 user-level file servers will need to be fixed to run on the new system.
539 A full description of what user-level servers must do to provide service with
540 the new 9P is beyond the scope of this paper.
541 Your best source of information is section 5 of the manual,
542 combined with study of a few examples.
543 .CW /sys/src/cmd/ramfs.c
544 is a simple example; it has a counterpart
545 .CW /sys/src/lib9p/ramfs.c
546 that implements the same service using the new
550 That said, it's worth summarizing what to watch for when converting a file server.
553 message is gone, and there is a now a
555 message that is exchanged at the start of a connection to establish
556 the version of the protocol to use (there's only one at the moment, identified by
559 and what the maximum message size will be.
560 This negotiation makes it easier to handle 9P encapsulation, such as with
562 and also permits larger message sizes when appropriate.
564 If your server wants to authenticate, it will need to implement an authentication file
567 message; otherwise it should return a helpful error string to the
569 request to signal that authentication is not required.
573 and directory reads will require some changes but they should not be fundamental.
574 Be aware that seeking on directories is forbidden, so it is fine if you disregard the
575 file offset when implementing directory reads; this makes it a little easier to handle
576 the variable-length entries.
577 You should still never return a partial directory entry; if the I/O count is too small
578 to return even one entry, you should return two bytes containing the byte count
579 required to represent the next entry in the directory.
580 User code can use this value to formulate a retry if it desires.
582 DIAGNOSTICS section of
584 for a description of this process.
586 The trickiest part of updating a file server is that the
590 messages have been merged into a single message, a sort of `clone-multiwalk'.
591 The new message, still called
593 proposes a sequence of file name elements to be evaluated using a possibly
595 The return message contains the qids of the files reached by
596 walking to the sequential elements.
597 If all the elements can be walked, the fid will be cloned if requested.
598 If a non-zero number of elements are requested, but none
599 can be walked, an error should be returned.
600 If only some can be walked, the fid is not cloned, the original fid is left
601 where it was, and the returned
603 message should contain the partial list of successfully reached qids.
606 for a full description.
projects / plan9front.git / blob