1# What is XNU? 2 3XNU kernel is part of the Darwin operating system for use in macOS and iOS operating systems. XNU is an acronym for X is Not Unix. 4XNU is a hybrid kernel combining the Mach kernel developed at Carnegie Mellon University with components from FreeBSD and a C++ API for writing drivers called IOKit. 5XNU runs on x86_64 and ARM64 for both single processor and multi-processor configurations. 6 7## The XNU Source Tree 8 9* `config` - configurations for exported apis for supported architecture and platform 10* `SETUP` - Basic set of tools used for configuring the kernel, versioning and kextsymbol management. 11* `EXTERNAL_HEADERS` - Headers sourced from other projects to avoid dependency cycles when building. These headers should be regularly synced when source is updated. 12* `libkern` - C++ IOKit library code for handling of drivers and kexts. 13* `libsa` - kernel bootstrap code for startup 14* `libsyscall` - syscall library interface for userspace programs 15* `libkdd` - source for user library for parsing kernel data like kernel chunked data. 16* `makedefs` - top level rules and defines for kernel build. 17* `osfmk` - Mach kernel based subsystems 18* `pexpert` - Platform specific code like interrupt handling, atomics etc. 19* `security` - Mandatory Access Check policy interfaces and related implementation. 20* `bsd` - BSD subsystems code 21* `tools` - A set of utilities for testing, debugging and profiling kernel. 22 23## How to Build XNU 24 25### Building a `DEVELOPMENT` Kernel 26 27The xnu make system can build kernel based on `KERNEL_CONFIGS` & `ARCH_CONFIGS` variables as arguments. 28Here is the syntax: 29 30```text 31make SDKROOT=<sdkroot> ARCH_CONFIGS=<arch> KERNEL_CONFIGS=<variant> 32``` 33 34Where: 35 36* `<sdkroot>`: path to macOS SDK on disk. (defaults to `/`) 37* `<variant>`: can be `debug`, `development`, `release`, `profile` and configures compilation flags and asserts throughout kernel code. 38* `<arch>`: can be valid arch to build for. (E.g. `X86_64`) 39 40To build a kernel for the same architecture as running OS, just type 41 42```text 43make SDKROOT=macosx.internal 44``` 45 46Additionally, there is support for configuring architectures through `ARCH_CONFIGS` and kernel configurations with `KERNEL_CONFIGS`. 47 48```text 49make SDKROOT=macosx.internal ARCH_CONFIGS=X86_64 KERNEL_CONFIGS=DEVELOPMENT 50make SDKROOT=macosx.internal ARCH_CONFIGS=X86_64 KERNEL_CONFIGS="RELEASE DEVELOPMENT DEBUG" 51``` 52 53> Note: By default, the architecture is set to the build machine's architecture, and the default kernel config is set to build for `DEVELOPMENT`. 54 55This will also create a bootable image, kernel.[config], and a kernel binary 56with symbols, kernel.[config].unstripped. 57 58To install the kernel into a DSTROOT, use the `install_kernels` target: 59 60```text 61make install_kernels DSTROOT=/tmp/xnu-dst 62``` 63 64For a more satisfying kernel debugging experience, with access to all 65local variables and arguments, but without all the extra check of the 66DEBUG kernel, add something like the following to your make command: 67 68```text 69CFLAGS_DEVELOPMENTARM64="-O0 -g -DKERNEL_STACK_MULTIPLIER=2" 70CXXFLAGS_DEVELOPMENTARM64="-O0 -g -DKERNEL_STACK_MULTIPLIER=2" 71``` 72 73Remember to replace `DEVELOPMENT` and `ARM64` with the appropriate build and platform. 74 75> Extra Flags: You can pass additional flags to the C compiler at the command line with the `EXTRA_CFLAGS` build setting. These flags are appended to the base `CFLAGS`, and the default value for the setting is an empty string. 76> 77> This setting allows you to e.g. selectively turn on debugging code that is guarded by a preprocessor macro. Example usage... 78> 79> ```text 80> make SDKROOT=macosx.internal PRODUCT_CONFIGS=j314s 81> EXTRA_CFLAGS='-DKERNEL_STACK_MULTIPLIER=2' 82> ``` 83 84 85* To build with RELEASE kernel configuration 86 87 ```text 88 make KERNEL_CONFIGS=RELEASE SDKROOT=/path/to/SDK 89 ``` 90 91### Building FAT Kernel Binary 92 93Define architectures in your environment or when running a make command. 94 95```text 96make ARCH_CONFIGS="X86_64" exporthdrs all 97``` 98 99### Other Makefile Options 100 101* $ make MAKEJOBS=-j8 # this will use 8 processes during the build. The default is 2x the number of active CPUS. 102* $ make -j8 # the standard command-line option is also accepted 103* $ make -w # trace recursive make invocations. Useful in combination with VERBOSE=YES 104* $ make BUILD_LTO=0 # build without LLVM Link Time Optimization 105* $ make BOUND_CHECKS=0 # disable -fbound-attributes for this build 106* $ make REMOTEBUILD=user@remotehost # perform build on remote host 107 108The XNU build system can optionally output color-formatted build output. To enable this, you can either 109set the `XNU_LOGCOLORS` environment variable to `y`, or you can pass `LOGCOLORS=y` to the make command. 110 111### Customize the XNU Version 112 113The xnu version is derived from the SDK or KDK by reading the `CFBundleVersion` 114of their `System/Library/Extensions/System.kext/Info.plist` file. 115This can be customized by setting the `RC_DARWIN_KERNEL_VERSION` variable in 116the environment or on the `make` command line. 117 118 119See doc/xnu_version.md for more details. 120 121### Debug Information Formats 122 123By default, a DWARF debug information repository is created during the install phase; this is a "bundle" named kernel.development.\<variant>.dSYM 124To select the older STABS debug information format (where debug information is embedded in the kernel.development.unstripped image), set the BUILD_STABS environment variable. 125 126```sh 127export BUILD_STABS=1 128make 129``` 130 131 132## Building KernelCaches 133 134To test the xnu kernel, you need to build a kernelcache that links the kexts and 135kernel together into a single bootable image. 136To build a kernelcache you can use the following mechanisms: 137 138* Using automatic kernelcache generation with `kextd`. 139 The kextd daemon keeps watching for changing in `/System/Library/Extensions` directory. 140 So you can setup new kernel as 141 142 ```text 143 cp BUILD/obj/DEVELOPMENT/X86_64/kernel.development /System/Library/Kernels/ 144 touch /System/Library/Extensions 145 ps -e | grep kextd 146 ``` 147 148* Manually invoking `kextcache` to build new kernelcache. 149 150 ```text 151 kextcache -q -z -a x86_64 -l -n -c /var/tmp/kernelcache.test -K /var/tmp/kernel.test /System/Library/Extensions 152 ``` 153 154 155## Booting a KernelCache on a Target machine 156 157The development kernel and iBoot supports configuring boot arguments so that we can safely boot into test kernel and, if things go wrong, safely fall back to previously used kernelcache. 158Following are the steps to get such a setup: 159 1601. Create kernel cache using the kextcache command as `/kernelcache.test` 1612. Copy exiting boot configurations to alternate file 162 163 ```sh 164 cp /Library/Preferences/SystemConfiguration/com.apple.Boot.plist /next_boot.plist 165 ``` 166 1673. Update the kernelcache and boot-args for your setup 168 169 ```sh 170 plutil -insert "Kernel Cache" -string "kernelcache.test" /next_boot.plist 171 plutil -replace "Kernel Flags" -string "debug=0x144 -v kernelsuffix=test " /next_boot.plist 172 ``` 173 1744. Copy the new config to `/Library/Preferences/SystemConfiguration/` 175 176 ```sh 177 cp /next_boot.plist /Library/Preferences/SystemConfiguration/boot.plist 178 ``` 179 1805. Bless the volume with new configs. 181 182 ```text 183 sudo -n bless --mount / --setBoot --nextonly --options "config=boot" 184 ``` 185 186 The `--nextonly` flag specifies that use the `boot.plist` configs only for one boot. 187 So if the kernel panic's you can easily power reboot and recover back to original kernel. 188 189 190## Creating tags and cscope 191 192Set up your build environment and from the top directory, run: 193 194 make tags # this will build ctags and etags on a case-sensitive volume, only ctags on case-insensitive 195 make TAGS # this will build etags 196 make cscope # this will build cscope database 197 198## Installing New Header Files from XNU 199 200XNU installs header files at the following locations - 201 202 a. $(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers 203 b. $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders 204 c. $(DSTROOT)/usr/include/ 205 d. $(DSTROOT)/usr/local/include/ 206 e. $(DSTROOT)/System/DriverKit/usr/include/ 207 f. $(DSTROOT)/System/Library/Frameworks/IOKit.framework/Headers 208 g. $(DSTROOT)/System/Library/Frameworks/IOKit.framework/PrivateHeaders 209 h. $(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders 210 211`Kernel.framework` is used by kernel extensions.\ 212The `System.framework`, `/usr/include` and `/usr/local/include` are used by user level applications. \ 213`IOKit.framework` is used by IOKit userspace clients. \ 214`/System/DriverKit/usr/include` is used by userspace drivers. \ 215The header files in framework's `PrivateHeaders` are only available for **Apple Internal Development**. 216 217The directory containing the header file should have a Makefile that 218creates the list of files that should be installed at different locations. 219If you are adding the first header file in a directory, you will need to 220create Makefile similar to `xnu/bsd/sys/Makefile`. 221 222Add your header file to the correct file list depending on where you want 223to install it. The default locations where the header files are installed 224from each file list are - 225 226 a. `DATAFILES` : To make header file available in user level - 227 `$(DSTROOT)/usr/include` 228 `$(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders` 229 230 b. `DRIVERKIT_DATAFILES` : To make header file available to DriverKit userspace drivers - 231 `$(DSTROOT)/System/DriverKit/usr/include` 232 233 c. `PRIVATE_DATAFILES` : To make header file available to Apple internal in 234 user level - 235 `$(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders` 236 237 d. `EMBEDDED_PRIVATE_DATAFILES` : To make header file available in user 238 level for macOS as `EXTRA_DATAFILES`, but Apple internal in user level 239 for embedded OSes as `EXTRA_PRIVATE_DATAFILES` - 240 `$(DSTROOT)/usr/include` (`EXTRA_DATAFILES`) 241 `$(DSTROOT)/usr/local/include` (`EXTRA_PRIVATE_DATAFILES`) 242 243 e. `KERNELFILES` : To make header file available in kernel level - 244 `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers` 245 `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders` 246 247 f. `PRIVATE_KERNELFILES` : To make header file available to Apple internal 248 for kernel extensions - 249 `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders` 250 251 g. `MODULEMAPFILES` : To make module map file available in user level - 252 `$(DSTROOT)/usr/include` 253 254 h. `PRIVATE_MODULEMAPFILES` : To make module map file available to Apple 255 internal in user level - 256 `$(DSTROOT)/usr/local/include` 257 258 i. `LIBCXX_DATAFILES` : To make header file available to in-kernel libcxx clients: 259 `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders/kernel_sdkroot` 260 261 j. `EXCLAVEKIT_DATAFILES` : To make header file available to Apple internal 262 ExclaveKit SDK - 263 `$(DSTROOT)/System/ExclaveKit/usr/include` 264 265 k. `EXCLAVECORE_DATAFILES` : To make header file available to Apple internal 266 ExclaveCore SDK - 267 `$(DSTROOT)/System/ExclaveCore/usr/include` 268 269The Makefile combines the file lists mentioned above into different 270install lists which are used by build system to install the header files. There 271are two types of install lists: machine-dependent and machine-independent. 272These lists are indicated by the presence of `MD` and `MI` in the build 273setting, respectively. If your header is architecture-specific, then you should 274use a machine-dependent install list (e.g. `INSTALL_MD_LIST`). If your header 275should be installed for all architectures, then you should use a 276machine-independent install list (e.g. `INSTALL_MI_LIST`). 277 278If the install list that you are interested does not exist, create it 279by adding the appropriate file lists. The default install lists, its 280member file lists and their default location are described below - 281 282a. `INSTALL_MI_LIST`, `INSTALL_MODULEMAP_MI_LIST` : Installs header and module map 283 files to a location that is available to everyone in user level. 284 Locations - 285 $(DSTROOT)/usr/include 286 Definition - 287 INSTALL_MI_LIST = ${DATAFILES} 288 INSTALL_MODULEMAP_MI_LIST = ${MODULEMAPFILES} 289 290b. `INSTALL_DRIVERKIT_MI_LIST` : Installs header file to a location that is 291 available to DriverKit userspace drivers. 292 Locations - 293 $(DSTROOT)/System/DriverKit/usr/include 294 Definition - 295 INSTALL_DRIVERKIT_MI_LIST = ${DRIVERKIT_DATAFILES} 296 297c. `INSTALL_MI_LCL_LIST`, `INSTALL_MODULEMAP_MI_LCL_LIST` : Installs header and 298 module map files to a location that is available for Apple internal in user level. 299 Locations - 300 $(DSTROOT)/usr/local/include 301 Definition - 302 INSTALL_MI_LCL_LIST = 303 INSTALL_MODULEMAP_MI_LCL_LIST = ${PRIVATE_MODULEMAPFILES} 304 305d. `INSTALL_IF_MI_LIST` : Installs header file to location that is available 306 to everyone for IOKit userspace clients. 307 Locations - 308 $(DSTROOT)/System/Library/Frameworks/IOKit.framework/Headers 309 Definition - 310 INSTALL_IF_MI_LIST = ${DATAFILES} 311 312e. `INSTALL_IF_MI_LCL_LIST` : Installs header file to location that is 313 available to Apple internal for IOKit userspace clients. 314 Locations - 315 $(DSTROOT)/System/Library/Frameworks/IOKit.framework/PrivateHeaders 316 Definition - 317 INSTALL_IF_MI_LCL_LIST = ${DATAFILES} ${PRIVATE_DATAFILES} 318 319f. `INSTALL_SF_MI_LCL_LIST` : Installs header file to a location that is available 320 for Apple internal in user level. 321 Locations - 322 $(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders 323 Definition - 324 INSTALL_SF_MI_LCL_LIST = ${DATAFILES} ${PRIVATE_DATAFILES} 325 326g. `INSTALL_KF_MI_LIST` : Installs header file to location that is available 327 to everyone for kernel extensions. 328 Locations - 329 $(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers 330 Definition - 331 INSTALL_KF_MI_LIST = ${KERNELFILES} 332 333h. `INSTALL_KF_MI_LCL_LIST` : Installs header file to location that is 334 available for Apple internal for kernel extensions. 335 Locations - 336 $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders 337 Definition - 338 INSTALL_KF_MI_LCL_LIST = ${KERNELFILES} ${PRIVATE_KERNELFILES} 339 340i. `EXPORT_MI_LIST` : Exports header file to all of xnu (bsd/, osfmk/, etc.) 341 for compilation only. Does not install anything into the SDK. 342 Definition - 343 EXPORT_MI_LIST = ${KERNELFILES} ${PRIVATE_KERNELFILES} 344 345j. `INSTALL_KF_LIBCXX_MI_LIST` : Installs header file for in-kernel libc++ support. 346 Locations - 347 $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders/kernel_sdkroot 348 Definition - 349 INSTALL_KF_LIBCXX_MI_LIST = ${LIBCXX_DATAFILES} 350 351k. `INSTALL_EXCLAVEKIT_MI_LIST` : Installs header file to location that is 352 available for Apple internal for ExclaveKit. 353 Locations - 354 $(DSTROOT)/System/ExclaveKit/usr/include 355 Definition - 356 INSTALL_EXCLAVEKIT_MI_LIST = ${EXCLAVEKIT_DATAFILES} 357 358l. `INSTALL_EXCLAVECORE_MI_LIST` : Installs header file to location that is 359 available for Apple internal for ExclaveCore. 360 Locations - 361 $(DSTROOT)/System/ExclaveCore/usr/include 362 Definition - 363 INSTALL_EXCLAVECORE_MI_LIST = ${EXCLAVECORE_DATAFILES} 364 365If you want to install the header file in a sub-directory of the paths 366described in (1), specify the directory name using two variables 367`INSTALL_MI_DIR` and `EXPORT_MI_DIR` as follows - 368 369```text 370INSTALL_MI_DIR = dirname 371EXPORT_MI_DIR = dirname 372``` 373 374If you want to install the module map file in a sub-directory, specify the 375directory name using the variable `INSTALL_MODULEMAP_MI_DIR` as follows - 376 377```text 378INSTALL_MODULEMAP_MI_DIR = dirname 379``` 380 381A single header file can exist at different locations using the steps 382mentioned above. However it might not be desirable to make all the code 383in the header file available at all the locations. For example, you 384want to export a function only to kernel level but not user level. 385 386 You can use C language's pre-processor directive (#ifdef, #endif, #ifndef) 387 to control the text generated before a header file is installed. The kernel 388 only includes the code if the conditional macro is TRUE and strips out 389 code for FALSE conditions from the header file. 390 391 Some pre-defined macros and their descriptions are - 392 3931. `PRIVATE` : If defined, enclosed definitions are considered System 394Private Interfaces. These are visible within xnu and 395exposed in user/kernel headers installed within the AppleInternal 396"PrivateHeaders" sections of the System and Kernel frameworks. 3972. `KERNEL_PRIVATE` : If defined, enclosed code is available to all of xnu 398kernel and Apple internal kernel extensions and omitted from user 399headers. 4003. `BSD_KERNEL_PRIVATE` : If defined, enclosed code is visible exclusively 401within the xnu/bsd module. 4024. `MACH_KERNEL_PRIVATE`: If defined, enclosed code is visible exclusively 403within the xnu/osfmk module. 4045. `XNU_KERNEL_PRIVATE`: If defined, enclosed code is visible exclusively 405within xnu. 4066. `KERNEL` : If defined, enclosed code is available within xnu and kernel 407 extensions and is not visible in user level header files. Only the 408 header files installed in following paths will have the code - 409 410 ```text 411 $(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers 412 $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders 413 ``` 414 4157. `DRIVERKIT`: If defined, enclosed code is visible exclusively in the 416DriverKit SDK headers used by userspace drivers. 4178. `EXCLAVEKIT`: If defined, enclosed code is visible exclusively in the 418ExclaveKit SDK headers. 4199. `EXCLAVECORE`: If defined, enclosed code is visible exclusively in the 420ExclaveCore SDK headers. 421 422## Module map file name convention 423 424In the simple case, a subdirectory of `usr/include` or `usr/local/include` 425can be represented by a standalone module. Where this is the case, set 426`INSTALL_MODULEMAP_MI_DIR` to `INSTALL_MI_DIR` and install a `module.modulemap` 427file there. `module.modulemap` is used even for private modules in 428`usr/local/include`; `module.private.modulemap` is not used. Caveat: in order 429to stay in the simple case, the module name needs to be exactly the same as 430the directory name. If that's not possible, then the following method will 431need to be applied. 432 433`xnu` contributes to the modules defined in CoreOSModuleMaps by installing 434module map files that are sourced from `usr/include/module.modulemap` and 435`usr/local/include/module.modulemap`. The naming convention for the `xnu` 436module map files are as follows. 437 438a. Ideally the module map file covers an entire directory. A module map 439 file covering `usr/include/a/b/c` would be named `a_b_c.modulemap`. 440 `usr/local/include/a/b/c` would be `a_b_c_private.modulemap`. 441b. Some headers are special and require their own module. In that case, 442 the module map file would be named after the module it defines. 443 A module map file defining the module `One.Two.Three` would be named 444 `one_two_three.modulemap`. 445 446## Conditional Compilation 447 448`xnu` offers the following mechanisms for conditionally compiling code: 449 4501. *CPU Characteristics* If the code you are guarding has specific 451 characterstics that will vary only based on the CPU architecture being 452 targeted, use this option. Prefer checking for features of the 453 architecture (e.g. `__LP64__`, `__LITTLE_ENDIAN__`, etc.). 4542. *New Features* If the code you are guarding, when taken together, 455 implements a feature, you should define a new feature in `config/MASTER` 456 and use the resulting `CONFIG` preprocessor token (e.g. for a feature 457 named `config_virtual_memory`, check for `#if CONFIG_VIRTUAL_MEMORY`). 458 This practice ensures that existing features may be brought to other 459 platforms by simply changing a feature switch. 4603. *Existing Features* You can use existing features if your code is 461 strongly tied to them (e.g. use `SECURE_KERNEL` if your code implements 462 new functionality that is exclusively relevant to the trusted kernel and 463 updates the definition/understanding of what being a trusted kernel means). 464 465It is recommended that you avoid compiling based on the target platform. `xnu` 466does not define the platform macros from `TargetConditionals.h` 467(`TARGET_OS_OSX`, `TARGET_OS_IOS`, etc.). 468 469 470## Debugging XNU 471 472By default, the kernel reboots in the event of a panic. 473This behavior can be overriden by the `debug` boot-arg -- `debug=0x14e` will cause a panic to wait for a debugger to attach. 474To boot a kernel so it can be debugged by an attached machine, override the `kdp_match_name` boot-arg with the appropriate `ifconfig` interface. 475Ethernet, Thunderbolt, and serial debugging are supported, depending on the hardware. 476 477Use LLDB to debug the kernel: 478 479```text 480xcrun -sdk macosx lldb <path-to-unstripped-kernel> 481(lldb) gdb-remote [<host-ip>:]<port> 482``` 483 484The debug info for the kernel (dSYM) comes with a set of macros to support kernel debugging. 485To load these macros automatically when attaching to the kernel, add the following to `~/.lldbinit`: 486 487```text 488settings set target.load-script-from-symbol-file true 489``` 490 491`tools/lldbmacros` contains the source for these commands. 492See the README in that directory for their usage, or use the built-in LLDB help with: 493 494```text 495(lldb) help showcurrentstacks 496``` 497 498