1# String handling in xnu 2 3xnu implements most POSIX C string functions, including the inherited subset of 4standard C string functions. Unfortunately, poor design choices have made many 5of these functions, including the more modern `strl` functions, confusing or 6unsafe. In addition, the advent of -fbounds-safety support in xnu is forcing 7some string handling practices to be revisited. This document explains the 8failings of POSIX C string functions, xnu's `strbuf` functions, and their 9intersection with the -fbounds-safety C extension. 10 11## The short-form guidance 12 13* Use `strbuf*` when you have the length for all the strings; 14* use `strl*` when you have the length of _one_ string, and the other is 15 guaranteed to be NUL-terminated; 16* use `str*` when you don't have the length for any of the strings, and they 17 are all guaranteed to be NUL-terminated; 18* stop using `strn*` functions. 19 20# The problems with string functions 21 22POSIX string handling functions come in many variants: 23 24* `str` functions (strlen, strcat, etc), unsafe for writing; 25* `strn` functions (strnlen, strncat, etc), unsafe for writing; 26* `strl` functions (strlcpy, strlcat, etc), safe but easily misunderstood. 27 28`str` functions for writing (`strcpy`, `strcat`, etc) are **all** unsafe 29because they don't care about the bounds of the output buffer. Most or all of 30these functions have been deprecated or outright removed from xnu. You should 31never use `str` functions to write to strings. Functions that simply read 32strings (`strlen`, `strcmp`, `strchr`, etc) are generally found to be safe 33because there is no confusion that their input must be NUL-terminated and there 34is no danger of writing out of bounds (out of not writing at all). 35 36`strn` functions for writing (`strncpy`, `strncat`, etc) are **all** unsafe. 37`strncpy` doesn't NUL-terminate the output buffer, and `strncat` doesn't accept 38a length for the output buffer. **All** new string buffers should include space 39for a NUL terminator. `strn` functions for reading (`strncmp`, `strnlen`) are 40_generally_ safe, but `strncmp` can cause confusion over which string is bound 41by the given size. In extreme cases, this can create information disclosure 42bugs or stability issues. 43 44`strl` functions, in POSIX, only come in writing variants, and they always 45NUL-terminate their output. This makes the writing part safe. (xnu adds `strl` 46comparison functions, which do no writing and are also safe.) However, these 47functions assume the output pointer is a buffer and the input is a NUL- 48terminated string. Because of coexistence with `strn` functions that make no 49such assumption, this mental model isn't entirely adopted by many users. For 50instance, the following code is buggy: 51 52```c 53char output[4]; 54char input[8] = "abcdefgh"; /* not NUL-terminated */ 55strlcpy(output, input, sizeof(output)); 56``` 57 58`strlcpy` returns the length of the input string; in xnu's implementation, 59literally by calling `strlen(input)`. Even though only 3 characters are written 60to `output` (plus a NUL), `input` is read until reaching a NUL character. This 61is always a problem from the perspective of memory disclosures, and in some 62cases, it can also lead to stability issues. 63 64# Changes with -fbounds-safety 65 66When enabling -fbounds-safety, character buffers and NUL-terminated strings are 67two distinct types, and they do not implicitly convert to each other. This 68prevents confusing the two in the way that is problematic with `strlcpy`, for 69instance. However, it creates new problems: 70 71* What is the correct way to transform a character buffer into a NUL-terminated 72 string? 73* When -fbounds-safety flags that the use of a string function was improper, 74 what is the solution? 75 76The most common use of character buffers is to build a string, and then this 77string is passed without bounds as a NUL-terminated string to downstream users. 78-fbounds-safety and XNU enshrine this practice with the following additions: 79 80* `tsnprintf`: like `snprintf`, but it returns a NUL-terminated string; 81* `strbuf` functions, explicitly accepting character buffers and a distinct 82 count for each: 83 * `strbuflen(buffer, length)`: like `strnlen`; 84 * `strbufcmp(a, alen, b, len)`: like `strcmp`; 85 * `strbufcasecmp(a, alen, b, blen)`: like `strcasecmp`; 86 * `strbufcpy(a, alen, b, blen)`: like `strlcpy` but returns `a` as a NUL- 87 terminated string; 88 * `strbufcat(a, alen, b, blen)`: like `strlcat` but returns `a` as a NUL- 89 terminated string; 90* `strl` (new) functions, accepting _one_ character buffer of a known size and 91 _one_ NUL-terminated string: 92 * `strlcmp(a, b, alen)`: like `strcmp`; 93 * `strlcasecmp(a, b, alen)`: like `strcasecmp`. 94 95`strbuf` functions additionally all have overloads accepting character arrays 96in lieu of a pointer+length pair: `strbuflen(array)`, `strbufcmp(a, b)`, 97`strbufcasecmp(a, b)`, `strbufcpy(a, b)`, `strbufcat(a, b)`. 98 99If the destination array of `strbufcpy` or `strbufcat` has a size of 0, they 100return NULL without doing anything else. Otherwise, the destination is always 101NUL-terminated and returned as a NUL-terminated string pointer. 102 103With -fbounds-safety enabled, the final operation modifying the character array 104should always return a NUL-terminated version of it. For instance, this plain C 105code: 106 107```c 108char thread_name[MAXTHREADNAMESIZE]; 109(void) snprintf(thread_name, sizeof(thread_name), 110 "dlil_input_%s", ifp->if_xname); 111thread_set_thread_name(inp->dlth_thread, thread_name); 112``` 113 114becomes: 115 116```c 117char thread_name_buf[MAXTHREADNAMESIZE]; 118const char *__null_terminated thread_name; 119thread_name = tsnprintf(thread_name_buf, sizeof(thread_name_buf), 120 "dlil_input_%s", ifp->if_xname); 121thread_set_thread_name(inp->dlth_thread, thread_name); 122``` 123 124Although `tsnprintf` and `strbuf` functions return a `__null_terminated` 125pointer to you for convenience, not all use cases are resolved by calling 126`tsnprintf` or `strbufcpy` once. As a quick reference, with -fbounds-safety 127enabled, you can use `__unsafe_null_terminated_from_indexable(p_start, p_nul)` 128to convert a character array to a `__null_terminated` string if you need to 129perform more manipulations. (`p_start` is a pointer to the first character, and 130`p_nul` is a pointer to the NUL character in that string.) For instance, if you 131build a string with successive calls to `scnprintf`, you would use 132`__unsafe_null_terminated_from_indexable` at the end of the sequence to get your 133NUL-terminated string pointer. 134 135# I have a choice between `strn*`, `strl*`, `strbuf*`. Which one do I use? 136 137You might come across cases where the same function in different families would 138seem like they all do the trick. For instance: 139 140```c 141struct foo { 142 char buf1[10]; 143 char buf2[16]; 144}; 145 146void bar(struct foo *f) { 147 /* how do I test whether buf1 and buf2 contain the same string? */ 148 if (strcmp(f->buf1, f->buf2) == 0) { /* ... */ } 149 if (strncmp(f->buf1, f->buf2, sizeof(f->buf1)) == 0) { /* ... */ } 150 if (strlcmp(f->buf1, f->buf2, sizeof(f->buf1)) == 0) { /* ... */ } 151 if (strbufcmp(f->buf1, f->buf2) == 0) { /* ... */ } 152} 153``` 154 155Without -fbounds-safety, these all work the same, but when you enable it, 156`strbufcmp` could be the only one that builds. If you do not have the privilege 157of -fbounds-safety to guide you to the best choice, as a rule of thumb, you 158should prefer APIs in the following order: 159 1601. `strbuf*` APIs; 1612. `strl*` APIs; 1623. `str*` APIs. 163 164That is, to implement `bar`, you have a choice of `strcmp`, `strncmp` and 165`strbufcmp`, and you should prefer `strbufcmp`. 166 167`strn` functions are **never** recommended. You should use `strbuflen` over 168`strnlen` (they do the same thing, but having a separate `strbuflen` function 169makes the guidance to avoid `strn` functions easier), and you should use 170`strbufcmp`, `strlcmp` or even `strcmp` over `strncmp` (depending on whether 171you know the length of each string, of just one, or of neither). 172