Commit dc1302d57779c918ad57e0a7873a613214c972ea
Commits[COMMIT BEGIN]commit dc1302d57779c918ad57e0a7873a613214c972ea Author: 0x4248 <[email protected]> Date: Sat Mar 28 21:16:40 2026 +0000 Add new pages diff --git a/doc/linux-ref/Makefile b/doc/linux-ref/Makefile index 8388f0e..65e450c 100644 --- a/doc/linux-ref/Makefile +++ b/doc/linux-ref/Makefile @@ -1,7 +1,7 @@ PANDOC ?= pandoc BUILDDIR = build CSS = docs/_static/whitepaper.css -TITLE = The 0x4248 Linux Reference Manual +TITLE = The 0x4248 Linux Systems Reference Manual SUBTITLE = A reference for GNU/Linux on x86-64 systems AUTHOR = 0x4248 BUILD_DATE = $(shell date -u +"%Y-%m-%d") diff --git a/doc/linux-ref/SOURCES b/doc/linux-ref/SOURCES new file mode 100644 index 0000000..a1f03ce --- /dev/null +++ b/doc/linux-ref/SOURCES @@ -0,0 +1,235 @@ +4248's Linux Reference Manual — Sources & References +====================================================== + +Primary sources are marked (*) — these should be checked first for any topic. +The kernel source tree is the ultimate ground truth for everything. + +──────────────────────────────────────────────────────────────────────────────── +KERNEL SOURCE AND OFFICIAL DOCUMENTATION +──────────────────────────────────────────────────────────────────────────────── + +(*) Linux kernel source tree + https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git + https://github.com/torvalds/linux + - arch/x86/entry/ syscall entry, vDSO + - arch/x86/mm/ page tables, KASLR + - kernel/sched/ CFS, RT, deadline schedulers + - kernel/signal.c signal delivery internals + - mm/ all memory management subsystems + - net/ full networking stack + - drivers/ all driver subsystems + - fs/ VFS and filesystem implementations + - security/ LSM, SELinux, AppArmor, Landlock + - include/uapi/ userspace-visible kernel headers + +(*) Kernel documentation (built from source, also online) + https://www.kernel.org/doc/html/latest/ + - admin-guide/kernel-parameters.txt all boot parameters + - admin-guide/mm/ memory management admin docs + - core-api/ core kernel API reference + - driver-api/ driver subsystem APIs + - filesystems/ per-filesystem documentation + - networking/ net stack documentation + - bpf/ eBPF subsystem + - trace/ ftrace, kprobes, perf events + - security/ LSM, SELinux, misc security + - process/ submitting patches, coding style + +(*) Linux man-pages project + https://www.kernel.org/doc/man-pages/ + https://man7.org/linux/man-pages/ + - Section 2: syscalls (definitive userspace interface reference) + - Section 4: device files (/dev/*) + - Section 5: /proc and /sys file formats + - Section 7: overviews (namespaces(7), capabilities(7), cgroups(7), etc.) + +(*) Linux Weekly News (LWN) + https://lwn.net + Best source for design rationale, historical context, and subsystem + introductions — search by topic before writing any major section. + + +──────────────────────────────────────────────────────────────────────────────── +BOOKS +──────────────────────────────────────────────────────────────────────────────── + +(*) Linux Kernel Development — Robert Love (3rd ed.) + Good overview of scheduling, memory, drivers, interrupts. + Note: covers ~2.6 era, cross-check against current source. + +(*) Understanding the Linux Kernel — Bovet & Cesati (3rd ed.) + Deep dive into process management, memory, VFS, I/O. + Note: 2.6 era, still accurate for conceptual structure. + +(*) Linux Device Drivers — Corbet, Rubini, Kroah-Hartman (3rd ed.) + https://lwn.net/Kernel/LDD3/ (free online) + Module interface, char/block drivers, memory, interrupts. + Note: 2.6 era, driver API has evolved — check kernel docs for updates. + + Linux Device Driver Development — John Madieu (2nd ed., 2022) + More current successor covering platform drivers, DT, modern APIs. + +(*) The Linux Programming Interface — Michael Kerrisk + Comprehensive userspace/syscall reference, proc/sys interfaces. + Covers namespaces, capabilities, signals, epoll, sockets in depth. + + BPF Performance Tools — Brendan Gregg (2019) + Covers eBPF, BCC, bpftrace, perf, flamegraphs end-to-end. + + Systems Performance — Brendan Gregg (2nd ed., 2020) + Methodology, perf counters, CPU/memory/disk/network observability. + + Computer Systems: A Programmer's Perspective — Bryant & O'Hallaron + Solid hardware/OS foundation: virtual memory, linking, I/O, concurrency. + + +──────────────────────────────────────────────────────────────────────────────── +ABI AND INTERFACE REFERENCES +──────────────────────────────────────────────────────────────────────────────── + +(*) Linux kernel user-space ABI docs + https://www.kernel.org/doc/html/latest/userspace-api/index.html + +(*) Linux allocated devices (major/minor numbers) + https://www.kernel.org/doc/html/latest/admin-guide/devices.html + +(*) POSIX / Single Unix Specification + https://pubs.opengroup.org/onlinepubs/9799919799/ + + GNU C Library (glibc) manual + https://www.gnu.org/software/libc/manual/ + + Filesystem Hierarchy Standard (FHS) + https://refspecs.linuxfoundation.org/fhs.shtml + + +──────────────────────────────────────────────────────────────────────────────── +ARCHITECTURE REFERENCES (x86_64 focus) +──────────────────────────────────────────────────────────────────────────────── + +(*) Intel 64 and IA-32 Architectures Software Developer Manuals + https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html + Vol 1: Basic architecture + Vol 2: Instruction set reference + Vol 3: System programming (paging, segmentation, interrupts, VMX) + +(*) AMD64 Architecture Programmer's Manual + https://www.amd.com/en/support/tech-docs/amd64-architecture-programmers-manual-volumes-1-5 + Vol 2: System programming (covers SVM for AMD virtualisation) + + x86_64 ABI supplement (System V ABI) + https://gitlab.com/x86-psABIs/x86-64-ABI + + +──────────────────────────────────────────────────────────────────────────────── +NETWORKING +──────────────────────────────────────────────────────────────────────────────── + +(*) iproute2 documentation and source + https://git.kernel.org/pub/scm/network/iproute2/iproute2.git + ip(8), tc(8), ss(8) man pages cover netlink usage clearly. + +(*) nftables wiki + https://wiki.nftables.org/ + + Netfilter project documentation + https://www.netfilter.org/documentation/ + + Linux Advanced Routing & Traffic Control (LARTC) HOWTO + https://lartc.org/howto/ + + +──────────────────────────────────────────────────────────────────────────────── +eBPF +──────────────────────────────────────────────────────────────────────────────── + +(*) BPF and XDP Reference Guide (Cilium) + https://docs.cilium.io/en/stable/bpf/ + +(*) Kernel BPF documentation + https://www.kernel.org/doc/html/latest/bpf/index.html + +(*) libbpf API docs + https://libbpf.readthedocs.io/ + + bpftrace reference guide + https://github.com/bpftrace/bpftrace/blob/master/docs/reference_guide.md + + Brendan Gregg's eBPF reference + https://ebpf.io/ + + +──────────────────────────────────────────────────────────────────────────────── +FILESYSTEMS +──────────────────────────────────────────────────────────────────────────────── + +(*) Ext4 documentation + https://www.kernel.org/doc/html/latest/filesystems/ext4/ + +(*) XFS documentation + https://www.kernel.org/doc/html/latest/filesystems/xfs/ + https://xfs.wiki.kernel.org/ + +(*) Btrfs documentation + https://btrfs.readthedocs.io/ + + VFS documentation + https://www.kernel.org/doc/html/latest/filesystems/vfs.html + + +──────────────────────────────────────────────────────────────────────────────── +SECURITY +──────────────────────────────────────────────────────────────────────────────── + +(*) Kernel self-protection project (KSPP) + https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project + +(*) SELinux notebook + https://github.com/SELinuxProject/selinux-notebook + + AppArmor documentation + https://gitlab.com/apparmor/apparmor/-/wikis/home + + Landlock documentation + https://landlock.io/ + https://www.kernel.org/doc/html/latest/userspace-api/landlock.html + + +──────────────────────────────────────────────────────────────────────────────── +HARDWARE / FIRMWARE +──────────────────────────────────────────────────────────────────────────────── + +(*) ACPI specification + https://uefi.org/specifications (ACPI 6.x) + +(*) PCI Express Base Specification + https://pcisig.com/specifications (requires free registration) + +(*) UEFI specification + https://uefi.org/specifications + + OSDev wiki (useful for low-level boot, ACPI, hardware interface details) + https://wiki.osdev.org/ + + +──────────────────────────────────────────────────────────────────────────────── +COMMUNITY AND SUPPLEMENTAL +──────────────────────────────────────────────────────────────────────────────── + + Kernel mailing lists (LKML and subsystem lists) + https://lore.kernel.org/ (searchable archive of all kernel lists) + + Linux kernel newbies + https://kernelnewbies.org/ (changelogs per release, concept explanations) + + Bootlin Elixir cross-referencer (browse kernel source with symbol links) + https://elixir.bootlin.com/linux/latest/source + + gregkh blog (Greg Kroah-Hartman, driver/USB/stable maintainer) + http://www.kroah.com/log/ + + Jonathan Corbet's LWN kernel coverage + https://lwn.net/Archives/ + + The /proc filesystem documentation + https://www.kernel.org/doc/html/latest/filesystems/proc.html diff --git a/doc/linux-ref/TODO b/doc/linux-ref/TODO new file mode 100644 index 0000000..ae951a9 --- /dev/null +++ b/doc/linux-ref/TODO @@ -0,0 +1,469 @@ +4248's Linux Reference Manual — Topic TODO +========================================== + +Legend: [ ] not started [~] in progress [x] done + +──────────────────────────────────────────────────────────────────────────────── +KERNEL +──────────────────────────────────────────────────────────────────────────────── + +[ ] kernel/ + [ ] boot/ + [ ] 10-bootloaders-and-handoff.md + [ ] GRUB2 / systemd-boot / EFISTUB + [ ] how the bootloader passes control to the kernel + [ ] multiboot2 header + [ ] 20-early-init.md + [ ] head_64.S entry point + [ ] decompression stage + [ ] early ioremap and fixmaps + [ ] 30-init-process.md + [ ] PID 1 from kernel perspective + [ ] kernel_init thread + [ ] initramfs / initrd mounting + [ ] 40-kernel-parameters.md + [ ] /proc/cmdline + [ ] Documentation/admin-guide/kernel-parameters.txt map + [ ] __setup() and early_param() macros + + [ ] cli-args/ [~ started: 10-kernel-command-line-overview.md] + [ ] 10-kernel-command-line-overview.md [~ draft] + [ ] 20-bootparam-reference.md + [ ] console=, root=, rw/ro, quiet, debug + [ ] module parameter passing (module.param=val) + [ ] 30-sysctl-vs-cmdline.md + + [ ] memory/ + [ ] 10-virtual-memory-layout.md + [ ] x86_64 canonical addresses + [ ] kernel vs userspace split (128 TB / 128 TB) + [ ] KASLR + [ ] 20-page-tables.md + [ ] PGD / P4D / PUD / PMD / PTE walk + [ ] huge pages (2 MB, 1 GB) + [ ] TLB shootdown + [ ] 30-slab-allocators.md + [ ] SLAB / SLUB / SLOB differences + [ ] kmalloc, vmalloc, kzalloc + [ ] /proc/slabinfo and slabtop + [ ] 40-mm-zones.md + [ ] ZONE_DMA, ZONE_DMA32, ZONE_NORMAL, ZONE_HIGHMEM + [ ] zone watermarks (min/low/high) + [ ] 50-oom-killer.md + [ ] oom_score and oom_adj + [ ] /proc/<pid>/oom_score_adj + [ ] oom_badness() selection logic + [ ] 60-huge-pages.md + [ ] THP (Transparent Huge Pages) + [ ] hugetlbfs + [ ] /sys/kernel/mm/hugepages/ + [ ] 70-cma.md + [ ] Contiguous Memory Allocator + [ ] dma_alloc_contiguous + + [ ] processes/ + [ ] 10-task-struct.md + [ ] key fields: pid, tgid, mm, fs, files, signal + [ ] copy_process() on fork + [ ] 20-scheduling.md + [ ] CFS fundamentals (vruntime, weight) + [ ] SCHED_FIFO / SCHED_RR / SCHED_DEADLINE + [ ] nr_running, load average + [ ] 30-namespaces.md + [ ] pid, net, mnt, uts, ipc, user, cgroup, time + [ ] /proc/<pid>/ns/ symlinks + [ ] unshare(1) and unshare(2) + [ ] 40-cgroups.md + [ ] cgroup v1 vs v2 hierarchy + [ ] cpu, memory, blkio, pids controllers + [ ] /sys/fs/cgroup/ + [ ] 50-signals.md + [ ] signal lifecycle and delivery + [ ] real-time signals SIGRTMIN–SIGRTMAX + [ ] /proc/<pid>/status SigBlk/SigPnd fields + [ ] 60-seccomp.md + [ ] seccomp-bpf filter + [ ] /proc/<pid>/status Seccomp field + [ ] 70-capabilities.md + [ ] capability sets: permitted/effective/inheritable/ambient/bounding + [ ] CAP_SYS_ADMIN and friends + [ ] getpcaps, capsh + + [ ] syscalls/ + [ ] 10-syscall-overview.md + [ ] syscall table (arch/x86/entry/syscalls/) + [ ] SYSCALL_DEFINE macros + [ ] vDSO and vsyscall + [ ] 20-key-syscalls.md + [ ] ioctl(2): encoding and direction bits + [ ] mmap(2): MAP_ANONYMOUS, MAP_SHARED, MAP_FIXED + [ ] epoll(7): level vs edge triggered + [ ] io_uring: SQ/CQ ring layout + [ ] 30-syscall-tracing.md + [ ] strace, ptrace internals + [ ] seccomp audit + [ ] perf trace + + [ ] modules/ + [ ] 10-module-anatomy.md + [ ] module_init / module_exit macros + [ ] THIS_MODULE, EXPORT_SYMBOL + [ ] 20-loading-and-linking.md + [ ] modprobe, insmod, rmmod + [ ] /proc/modules, lsmod + [ ] symbol versioning (MODVERSIONS) + [ ] 30-out-of-tree-builds.md + [ ] Kbuild for external modules + [ ] KERNELDIR / KDIR Makefile pattern + + [ ] drivers/ + [ ] 10-device-model.md + [ ] bus/device/driver trifecta + [ ] platform_driver, pci_driver registration + [ ] sysfs binding: /sys/bus/*/drivers/ + [ ] 20-char-devices.md + [ ] cdev_init, cdev_add + [ ] file_operations struct + [ ] major/minor allocation (alloc_chrdev_region) + [ ] 30-interrupt-handling.md + [ ] request_irq, free_irq + [ ] top half vs threaded IRQ (IRQF_THREAD) + [ ] /proc/interrupts + [ ] 40-dma.md + [ ] coherent vs streaming DMA + [ ] dma_map_single, dma_alloc_coherent + [ ] IOMMU and DMA remapping + + [ ] tracing-and-debug/ + [ ] 10-ftrace.md + [ ] /sys/kernel/debug/tracing/ + [ ] function_graph tracer + [ ] trace-cmd usage + [ ] 20-kprobes.md + [ ] kprobe / kretprobe + [ ] eBPF kprobe programs + [ ] 30-perf.md + [ ] perf stat, perf record, perf report + [ ] PMU events and raw perf_event_attr + [ ] perf annotate for hot paths + [ ] 40-kdump-and-kgdb.md + [ ] crash utility basics + [ ] /proc/vmcore + [ ] kgdb over serial / kgdb over ethernet (netpoll) + + +──────────────────────────────────────────────────────────────────────────────── +DEVICE FILES AND /dev +──────────────────────────────────────────────────────────────────────────────── + +[ ] filesystems-and-device-nodes/ [~ started: 01-filesystems-and-device-nodes.md] + [ ] 10-dev-overview.md [~ draft] + [ ] major/minor numbers + [ ] udev rules and mdev + [ ] 20-notable-char-devices.md + [ ] /dev/null, /dev/zero, /dev/full, /dev/urandom, /dev/random + [ ] /dev/mem, /dev/kmem (and their restrictions) + [ ] /dev/tty, /dev/ptmx, /dev/pts/* + [ ] /dev/input/event*, /dev/input/mouse* + [ ] /dev/fb0 (framebuffer), /dev/dri/card* (DRM) + [ ] /dev/kvm + [ ] /dev/loop*, /dev/mapper/* + [ ] /dev/watchdog + [ ] 30-block-devices.md + [ ] /dev/sd*, /dev/nvme*, /dev/mmcblk* + [ ] partition naming + [ ] /sys/block/ relationship + [ ] 40-udev.md + [ ] udev rules syntax (KERNEL==, ATTR{}, RUN+=) + [ ] udevadm monitor, udevadm info + [ ] persistent naming rules + + +──────────────────────────────────────────────────────────────────────────────── +PSEUDO FILESYSTEMS +──────────────────────────────────────────────────────────────────────────────── + +[ ] pseudo-fs/ + [ ] 10-procfs.md + [ ] /proc/self, /proc/<pid>/ tree + [ ] /proc/meminfo field-by-field + [ ] /proc/net/* (tcp, udp, unix, dev) + [ ] /proc/sys/ hierarchy (same as sysctl) + [ ] 20-sysfs.md + [ ] sysfs as kernel object mirror + [ ] attribute files and kobject model + [ ] power management: /sys/devices/ power/ + [ ] 30-debugfs.md + [ ] mounting debugfs + [ ] key subsystem nodes (drm, tracing, bdi) + [ ] 40-cgroup-fs.md + [ ] v1 multi-hierarchy vs v2 unified + [ ] delegating subtrees (cgroup.subtree_control) + [ ] 50-tmpfs-and-ramfs.md + [ ] tmpfs accounting and limits + [ ] /dev/shm as tmpfs + [ ] 60-bpffs.md + [ ] pinning eBPF maps and programs + [ ] /sys/fs/bpf/ + + +──────────────────────────────────────────────────────────────────────────────── +FILESYSTEMS +──────────────────────────────────────────────────────────────────────────────── + +[ ] filesystems/ + [ ] 10-vfs.md + [ ] inode, dentry, superblock, file objects + [ ] VFS operations tables + [ ] dcache and icache + [ ] 20-ext4.md + [ ] extents vs indirect blocks + [ ] journal modes (journal, ordered, writeback) + [ ] tune2fs knobs + [ ] 30-xfs.md + [ ] allocation groups + [ ] delayed allocation + [ ] xfs_info, xfs_repair + [ ] 40-btrfs.md + [ ] CoW and subvolumes + [ ] RAID modes + [ ] btrfs send/receive + [ ] 50-overlayfs.md + [ ] lower/upper/work/merged layout + [ ] container image layers + [ ] 60-fuse.md + [ ] userspace filesystem protocol + [ ] libfuse request handling loop + + +──────────────────────────────────────────────────────────────────────────────── +NETWORKING +──────────────────────────────────────────────────────────────────────────────── + +[ ] networking/ + [ ] 10-socket-internals.md + [ ] sock struct and sk_buff lifecycle + [ ] socket options (SO_*, TCP_*, IP_*) + [ ] send/recv buffer tuning + [ ] 20-netfilter.md + [ ] hook points: PREROUTING, INPUT, FORWARD, OUTPUT, POSTROUTING + [ ] nftables rule structure + [ ] conntrack: /proc/net/nf_conntrack + [ ] 30-tc.md + [ ] qdiscs: fq, fq_codel, pfifo_fast, tbf + [ ] tc filter with BPF classifier + [ ] XDP: rx hook before netdev + [ ] 40-network-namespaces.md + [ ] ip netns, veth pairs + [ ] vrf and routing tables per ns + [ ] 50-tun-tap.md + [ ] TUN/TAP devices + [ ] /dev/net/tun usage + [ ] userspace VPN plumbing + [ ] 60-netlink.md + [ ] NETLINK_ROUTE / NETLINK_SOCK_DIAG + [ ] rtnetlink attribute encoding + [ ] iproute2 internals using netlink + + +──────────────────────────────────────────────────────────────────────────────── +BLOCK I/O +──────────────────────────────────────────────────────────────────────────────── + +[ ] block/ + [ ] 10-io-stack.md + [ ] bio and request structs + [ ] I/O schedulers: none, mq-deadline, kyber, bfq + [ ] /sys/block/<dev>/queue/scheduler + [ ] 20-io-accounting.md + [ ] /proc/<pid>/io + [ ] blkio cgroup controller + [ ] iostat, iotop internals + [ ] 30-nvme.md + [ ] NVMe queue pairs (submission/completion) + [ ] nvme cli management + [ ] nvme namespaces + + +──────────────────────────────────────────────────────────────────────────────── +eBPF +──────────────────────────────────────────────────────────────────────────────── + +[ ] ebpf/ + [ ] 10-overview.md + [ ] BPF ISA: registers, instructions, maps + [ ] verifier constraints + [ ] JIT and interpreted modes + [ ] 20-program-types.md + [ ] kprobe, tracepoint, perf_event + [ ] XDP, TC, cgroup_skb + [ ] lsm, fentry/fexit + [ ] 30-maps.md + [ ] hash, array, ringbuf, perf_event_array + [ ] map pinning via bpffs + [ ] 40-bpftool.md + [ ] bpftool prog / map / net inspection + [ ] BTF and CO-RE + [ ] 50-libbpf-skeleton.md + [ ] skel-based user space loader pattern + + +──────────────────────────────────────────────────────────────────────────────── +BASH +──────────────────────────────────────────────────────────────────────────────── + +[ ] bash/ [~ started: 10-bash-overview.md] + [ ] 10-bash-overview.md [~ draft] + [ ] 20-expansions.md + [ ] brace, tilde, parameter, arithmetic, command sub, word splitting, glob + [ ] order of expansion stages + [ ] 30-parameter-internals.md + [ ] special vars: $*, $@, $#, $?, $!, $-, $$, $0 + [ ] namerefs (declare -n) + [ ] arrays and associative arrays + [ ] 40-io-redirection.md + [ ] fd duplication: 2>&1, >&, <&- + [ ] process substitution <() >() + [ ] here-doc and here-string + [ ] 50-job-control.md + [ ] fg, bg, disown, wait + [ ] /proc/<pid>/fdinfo/ under the hood + [ ] 60-builtins-vs-externals.md + [ ] why type/which/command differ + [ ] hash table and PATH walk + [ ] 70-readline-and-inputrc.md + [ ] 80-scripting-patterns.md + [ ] error handling (set -euo pipefail) + [ ] trap cleanup patterns + [ ] lockfile and singleton techniques + + +──────────────────────────────────────────────────────────────────────────────── +INIT AND SERVICE MANAGEMENT +──────────────────────────────────────────────────────────────────────────────── + +[ ] init/ + [ ] 10-systemd-units.md + [ ] service, socket, timer, mount, device unit types + [ ] dependency ordering (Wants, After, Requires) + [ ] systemctl status internals (cgroup tree) + [ ] 20-systemd-journal.md + [ ] structured fields + [ ] journald socket protocol + [ ] journalctl filtering on _COMM= _PID= etc + [ ] 30-boot-target-sequence.md + [ ] default.target chain + [ ] rescue.target and emergency.target + [ ] 40-socket-activation.md + [ ] LISTEN_FDS protocol + [ ] sd_notify(3) + + +──────────────────────────────────────────────────────────────────────────────── +SECURITY +──────────────────────────────────────────────────────────────────────────────── + +[ ] security/ + [ ] 10-lsm.md + [ ] LSM hook architecture + [ ] stacking order (security_hook_heads) + [ ] 20-selinux.md + [ ] type enforcement: subject/object/permission + [ ] policy modules and audit2allow + [ ] /sys/fs/selinux/ + [ ] 30-apparmor.md + [ ] profile syntax + [ ] aa-genprof, aa-logprof + [ ] 40-landlock.md + [ ] ruleset, filesystem and net access control + [ ] landlock_create_ruleset(2) + [ ] 50-kernel-hardening.md + [ ] SMEP, SMAP, CET, KPTI + [ ] KSPP config options + [ ] /proc/sys/kernel/kptr_restrict, dmesg_restrict + + +──────────────────────────────────────────────────────────────────────────────── +HARDWARE INTERFACES +──────────────────────────────────────────────────────────────────────────────── + +[ ] hardware/ + [ ] 10-pci-and-pcie.md + [ ] config space: vendor/device/BAR layout + [ ] lspci -vvv output walkthrough + [ ] /sys/bus/pci/devices/ attributes + [ ] 20-usb.md + [ ] transfer types: control, bulk, interrupt, isochronous + [ ] /sys/bus/usb/devices/ and usbfs + [ ] lsusb -v walkthrough + [ ] 30-acpi.md + [ ] DSDT / SSDT / FADT tables + [ ] acpidump + iasl decompile + [ ] /sys/firmware/acpi/ + [ ] 40-gpio.md + [ ] character device gpio (/dev/gpiochip*) + [ ] libgpiod and gpioinfo + [ ] 50-i2c-and-spi.md + [ ] /dev/i2c-*, i2c-tools + [ ] /dev/spidev*, spidev ioctls + [ ] 60-cpu-topology.md + [ ] /sys/devices/system/cpu/ layout + [ ] NUMA: /sys/devices/system/node/ + [ ] CPU hotplug + + +──────────────────────────────────────────────────────────────────────────────── +VIRTUAL MACHINES AND CONTAINERS +──────────────────────────────────────────────────────────────────────────────── + +[ ] virt/ + [ ] 10-kvm.md + [ ] /dev/kvm ioctls: KVM_CREATE_VM, KVM_CREATE_VCPU + [ ] VMCS / VMCB overview + [ ] EPT / NPT (nested page tables) + [ ] 20-containers.md + [ ] namespace + cgroup composition + [ ] /proc/<pid>/ns/ fd trick for re-entering + [ ] pivot_root vs chroot + [ ] 30-virtio.md + [ ] virtqueue split ring layout + [ ] virtio-blk, virtio-net, virtio-mem + [ ] vhost-user and DPDK path + + +──────────────────────────────────────────────────────────────────────────────── +TOOLING REFERENCE +──────────────────────────────────────────────────────────────────────────────── + +[ ] tooling/ + [ ] 10-binutils.md + [ ] objdump, readelf, nm, addr2line, size + [ ] ELF structure walkthrough + [ ] 20-gdb.md + [ ] kernel debugging with gdb + vmlinux + [ ] QEMU + gdb stub + [ ] crash scripting with gdb + [ ] 30-sparse-and-coccinelle.md + [ ] sparse annotations: __user, __iomem, __must_check + [ ] Coccinelle semantic patch basics + [ ] 40-kconfig-and-kbuild.md + [ ] Kconfig syntax: bool, tristate, depends on, select + [ ] Kbuild obj-y / obj-m mechanics + [ ] make menuconfig, allmodconfig, tinyconfig + + +──────────────────────────────────────────────────────────────────────────────── +APPENDIX +──────────────────────────────────────────────────────────────────────────────── + +[ ] appendix/ + [ ] 10-syscall-table.md + [ ] x86_64 syscall table (annotated) + [ ] 20-errno-reference.md + [ ] full errno list with common kernel context + [ ] 30-ioctl-encoding.md + [ ] _IO, _IOR, _IOW, _IOWR macros + [ ] direction and size bit layout + [ ] 40-glossary.md + [ ] kernel / systems jargon A–Z + diff --git a/doc/linux-ref/docs/_static/pdf-header.tex b/doc/linux-ref/docs/_static/pdf-header.tex index 82a1d42..56505af 100644 --- a/doc/linux-ref/docs/_static/pdf-header.tex +++ b/doc/linux-ref/docs/_static/pdf-header.tex @@ -13,6 +13,22 @@ }% } +% Running header with book title +\usepackage{fancyhdr} +\pagestyle{fancy} +\fancyhf{} +\fancyhead[L]{\small\textit{The 0x4248 Linux Systems Reference Manual}} +\fancyhead[R]{\small\thepage} +\fancyfoot{} +\renewcommand{\headrulewidth}{0.4pt} +% Keep plain style (chapter opening pages) consistent +\fancypagestyle{plain}{% + \fancyhf{} + \fancyhead[L]{\small\textit{The 0x4248 Linux Systems Reference Manual}} + \fancyhead[R]{\small\thepage} + \renewcommand{\headrulewidth}{0.4pt} +} + % Make \paragraph (h5) and \subparagraph (h6) block-level headings \makeatletter \renewcommand\paragraph{\@startsection{paragraph}{4}{\z@}% diff --git a/doc/linux-ref/docs/_static/whitepaper.css b/doc/linux-ref/docs/_static/whitepaper.css index 1729450..3933acb 100644 --- a/doc/linux-ref/docs/_static/whitepaper.css +++ b/doc/linux-ref/docs/_static/whitepaper.css @@ -33,3 +33,28 @@ pre, tt { font-family: "Liberation Mono", "DejaVu Sans Mono", monospace; } + +#title-block-header { + border-bottom: 2px solid #111111; + margin-bottom: 2rem; + padding-bottom: 1rem; +} + +#title-block-header h1.title { + font-size: 1.6rem; + font-weight: 700; + margin: 0 0 0.25rem 0; +} + +#title-block-header p.subtitle { + font-size: 1rem; + font-style: italic; + margin: 0 0 0.5rem 0; +} + +#title-block-header p.author, +#title-block-header p.date { + font-size: 0.85rem; + color: #555555; + margin: 0; +} diff --git a/doc/linux-ref/docs/chapters/00-frontmatter/01-intro.md b/doc/linux-ref/docs/chapters/00-frontmatter/01-intro.md index d2b5576..0f3bc1b 100644 --- a/doc/linux-ref/docs/chapters/00-frontmatter/01-intro.md +++ b/doc/linux-ref/docs/chapters/00-frontmatter/01-intro.md @@ -4,7 +4,10 @@ Welcome to the 0x4248 Linux Reference Manual. This manual is a very technical Linux Reference for practitioners working close to kernel internals, user-space interfaces, and Linux system behavior. -**This is NOT a beginner's manual.** +$$ +\color{red} +\text{\huge{\textbf{{This is NOT a beginners's manual.}}}} \\ +$$ If you are new to Linux, I strongly recommend starting with a more beginner-friendly resource, such as the @@ -91,4 +94,6 @@ this manual, including text, images, and other media. - [Arch Wiki](https://wiki.archlinux.org/) - [Linux From Scratch](http://www.linuxfromscratch.org/) - [syscall.sh](https://syscall.sh/) -- [man7.org](https://man7.org/) \ No newline at end of file +- [man7.org](https://man7.org/) + +Huge thanks to the [All Things Linux](https://discord.gg/linux) for the welcoming and helpful community. \ No newline at end of file diff --git a/doc/linux-ref/docs/chapters/01-kernel/01-the_kernel_basics.md b/doc/linux-ref/docs/chapters/01-kernel/01-the_kernel_basics.md index f10559c..7e5e973 100644 --- a/doc/linux-ref/docs/chapters/01-kernel/01-the_kernel_basics.md +++ b/doc/linux-ref/docs/chapters/01-kernel/01-the_kernel_basics.md @@ -198,7 +198,7 @@ which allows `make` to use all available cores for the build. I recommend using `ccache` to speed up subsequent builds. `ccache` is a compiler cache that stores the results of previous compilations and reuses them when the same -compilation is requested again. This can significan notly reduce the build time for +compilation is requested again. This can significantly speed up subsequent builds, especially when making small changes to the kernel configuration. To use `ccache`, you can set the `CC` environment variable to `ccache gcc` before diff --git a/doc/linux-ref/docs/chapters/01-kernel/02-kernel-source.md b/doc/linux-ref/docs/chapters/01-kernel/02-kernel-source.md index 6410b84..5fa93f0 100644 --- a/doc/linux-ref/docs/chapters/01-kernel/02-kernel-source.md +++ b/doc/linux-ref/docs/chapters/01-kernel/02-kernel-source.md @@ -6,7 +6,7 @@ understand. However, the base Linux kernel source code is relatively small, with the majority of the code being drivers and architecture-specific code. In the last section we discussed how the Linux kernel can be obtained and built. -Now lets read and go over the source code itself. +Now let's read and go over the source code itself. ## Layout of the source code @@ -94,5 +94,343 @@ then the actual code. authors. It also includes a history of changes to the file. 4. Line 11 is a macro definition and where the actual code starts. -The rest of this file contains allot of `#include` statements. This is because +The rest of this file contains a lot of `#include` statements. This is because this is one of the core kernel init files. + +Below in this file there will also be functions and variables defined. We won't +go over how to read and understand C code in this book, but if you are interested. +I strongly recommend the book +[The C Programming Language](https://en.wikipedia.org/wiki/The_C_Programming_Language) + +On several functions you may see the `__init` macro. This is a special macro +that tells the kernel that the function is only used during initialization. Once +the kernel has finished initializing, the memory used by these functions can be +freed up. This is because these functions are only needed once. + +To make a function a initialization function, you can simply add the macro to +the function definition. For example: + +```c +static int __init my_init_function(void) { + return 0; +} +``` + +### Kbuild and Makefiles + +The Linux kernel uses a custom build system known as Kbuild, which is built +on top of make. Due to the size of the codebase, the build system is split +across many smaller Makefiles distributed throughout the source tree. + +The top-level Makefile, located at the root of the kernel source, orchestrates +the overall build. It is responsible for invoking the build process and +descending into subdirectories. In practice, it is rarely modified directly. + +Most work is done in per-directory Makefiles (often referred to as Kbuild +files). These describe: + +- Which source files are compiled +- Whether they are built into the kernel or as modules +- Relationships between objects + +Kbuild operates recursively. Each directory contributes objects, which are +aggregated into: + +- `vmlinux` (the final kernel image) +- Loadable modules (`*.ko`) + +A typical entry: + +``` +obj-y += main.o +``` + +This instructs Kbuild to compile `main.c` and link it into the kernel image. + +Alternatively: + +``` +obj-m += my_driver.o +``` + + +This builds `my_driver.c` as a loadable kernel module. + +Kbuild works closely with the kernel configuration system (`Kconfig`). +Kconfig determines *what* is built, while Kbuild determines *how* it is built. + +### Kconfig + +Kconfig (Kernel Configuration) is the system used to manage kernel build +options. The kernel supports a wide range of hardware and features, so it is +not practical to build everything into a single image. Kconfig allows the user +to select exactly which components are included. + +Configuration options are defined across many `Kconfig` files throughout the +source tree. These files form a hierarchy, which is presented to the user via +interfaces such as `menuconfig`. + +The main `Kconfig` file at the root of the kernel source is minimal. It +primarily includes other `Kconfig` files from subdirectories, forming the full +configuration tree. + +*Kconfig* + +```c +mainmenu "Linux/$(ARCH) $(KERNELVERSION) Kernel Configuration" + +source "scripts/Kconfig.include" + +source "init/Kconfig" + +source "kernel/Kconfig.freezer" + +source "fs/Kconfig.binfmt" + +... +``` + +Lets take a look at a better `Kconfig` file, the one located in `lib/Kconfig.debug`. This is responsible for the kernel hacking and debugging options. + +*lib/Kconfig.debug* + +``` +# SPDX-License-Identifier: GPL-2.0-only +menu "Kernel hacking" + +menu "printk and dmesg options" + +config PRINTK_TIME + bool "Show timing information on printks" + depends on PRINTK + help + Selecting this option causes time stamps of the printk() + messages to be added to the output of the syslog() system + call and at the console. + + The timestamp is always recorded internally, and exported + to /dev/kmsg. This flag just specifies if the timestamp should + be included, not that the timestamp is recorded. + + The behavior is also controlled by the kernel command line + parameter printk.time=1. See Documentation/admin-guide/kernel-parameters.rst + +config PRINTK_CALLER + bool "Show caller information on printks" + depends on PRINTK + help + Selecting this option causes printk() to add a caller "thread id" (if + in task context) or a caller "processor id" (if not in task context) + to every message. + + This option is intended for environments where multiple threads + concurrently call printk() for many times, for it is difficult to + interpret without knowing where these lines (or sometimes individual + line which was divided into multiple lines due to race) came from. + + Since toggling after boot makes the code racy, currently there is + no option to enable/disable at the kernel command line parameter or + sysfs interface. + +config STACKTRACE_BUILD_ID + bool "Show build ID information in stacktraces" + depends on PRINTK + help + Selecting this option adds build ID information for symbols in + stacktraces printed with the printk format '%p[SR]b'. + + This option is intended for distros where debuginfo is not easily + accessible but can be downloaded given the build ID of the vmlinux or + kernel module where the function is located. + +config CONSOLE_LOGLEVEL_DEFAULT + int "Default console loglevel (1-15)" + range 1 15 + default "7" + help + Default loglevel to determine what will be printed on the console. + + Setting a default here is equivalent to passing in loglevel=<x> in + the kernel bootargs. loglevel=<x> continues to override whatever + value is specified here as well. + + Note: This does not affect the log level of un-prefixed printk() + usage in the kernel. That is controlled by the MESSAGE_LOGLEVEL_DEFAULT + option. +``` + +This can look complex compared to your traditional `JSON` or `YAML` configuration files, but it is actually quite simple if we break it down. + +- `menu "Kernel hacking"`: This defines a new menu in the menuconfig +- `menu "printk and dmesg options"`: This defines a submenu under the "Kernel hacking" menu +- `config PRINTK_TIME`: This defines a new configuration option called `PRINTK_TIME` however on build this will be converted to `CONFIG_PRINTK_TIME` +to prevent name clashes. +- `bool "Show timing information on printks"`: This specifies that the option is a boolean (true/false) and provides a description for it. +- `depends on PRINTK`: This specifies that this option can only be enabled if the `PRINTK` option is also enabled. +- `help`: This provides a help message that will be displayed when the user selects this option in the menuconfig interface. + +An `endmenu` is further down in the file to leave the submenu and return to the parent menu. + +Running the kernel currently will show a timestamp next to the logs: + +*QEMU* + +``` +[ 0.000000] Linux version 7.0.0-rc4-gee5389d2ec94 +[ 0.000000] Command line: console=ttyS0 +[ 0.000000] BIOS-provided physical RAM map: +``` + +Now lets disable the `PRINTK_TIME` option in menuconfig: + +*QEMU* + +``` +Linux version 7.0.0-rc4-g0f052016d8b4-dirty +Command line: console=ttyS0 +BIOS-provided physical RAM map: +``` + +Now you can see how the timestamps are gone. This is just one example of how +Kconfig allows you to easily configure the kernel and enable or disable features +as needed. + +Kconfig writes the configuration options to a file called `.config` in the root +of the kernel source. This file is then used by Kbuild. Editing this file is not +recommended, but we can inspect it to see how the options are stored. + +*.config* + +``` +# +# Automatically generated file; DO NOT EDIT. +# Linux/x86 7.0.0-rc4 Kernel Configuration +# +CONFIG_CC_VERSION_TEXT="gcc (Debian 14.2.0-19) 14.2.0" +CONFIG_CC_IS_GCC=y +CONFIG_GCC_VERSION=140200 +CONFIG_CLANG_VERSION=0 +CONFIG_AS_IS_GNU=y +...and so on +``` + +We can find for the `CONFIG_PRINTK_TIME` option: + +``` +# +# printk and dmesg options +# +# CONFIG_PRINTK_TIME is not set +# CONFIG_PRINTK_CALLER is not set +# CONFIG_STACKTRACE_BUILD_ID is not set +CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7 +``` + +You will notice it is not set and we can re-enable it by changing the line to: + +``` +CONFIG_PRINTK_TIME=y +``` + +Recompiling the kernel will then enable the timestamps again. + +#### Hello World Kconfig Example + +Now we know how the configuration system works, let's create a simple "Hello World" configuration option. + +Going back to the `lib/Kconfig.debug` file, we can add a new option at the start +of the file: + +*lib/Kconfig.debug* + +``` +# SPDX-License-Identifier: GPL-2.0-only +menu "Kernel hacking" + +menu "printk and dmesg options" + +config HELLO_WORLD + bool "Hello world" + depends on PRINTK + help + Demo option to show how config options work. Adds "Hello world!" to the + printk output during boot. +``` + +Now we should see: + +``` +Kernel hacking ---> + printk and dmesg options ---> + [ ] Hello world (NEW) +``` + +Now enable it, `CONFIG_HELLO_WORLD` now exists in the `.config` and we can make +a simple patch in the `printk` system to append "Hello world!" to the end of +every printk message. + + +Now go to this file: + +*kernel/printk/printk.c* + +```c +static bool printk_time = IS_ENABLED(CONFIG_PRINTK_TIME); +module_param_named(time, printk_time, bool, S_IRUGO | S_IWUSR); + +static size_t print_syslog(unsigned int level, char *buf) +{ + return sprintf(buf, "<%u>", level); +} + +static size_t print_time(u64 ts, char *buf) +{ + unsigned long rem_nsec = do_div(ts, 1000000000); + + return sprintf(buf, "[%5lu.%06lu]", + (unsigned long)ts, rem_nsec / 1000); +} +``` + +I know this is not the best way to add new features but we are only demonstrating +how to use Kconfig and the configuration system. + +*kernel/printk/printk.c* + +```c +static size_t print_time(u64 ts, char *buf) +{ + unsigned long rem_nsec = do_div(ts, 1000000000); + +#ifdef CONFIG_HELLO_WORLD + return sprintf(buf, "Hello World! [%5lu.%06lu]", + (unsigned long)ts, rem_nsec / 1000); +#else + return sprintf(buf, "[%5lu.%06lu]", + (unsigned long)ts, rem_nsec / 1000); +#endif +} +``` + +So now when on compile it will only keep the "Hello World!" string if `CONFIG_HELLO_WORLD` is enabled. + + +**Note**: You might want to re-enable the `PRINTK_TIME` option to see the effect +of this change, since in the last section we disabled it. + + +*QEMU* + +``` +Hello World! [ 0.000000] Linux version 7.0.0-rc4-g0f052016d8b4-dirty +Hello World! [ 0.000000] Command line: console=ttyS0 +Hello World! [ 0.000000] BIOS-provided physical RAM map: +Hello World! [ 0.000000] NX (Execute Disable) protection: active +Hello World! [ 0.000000] APIC: Static calls initialized +``` + +Now we see our "Hello World!" string in the printk output during boot. This is +just a simple example of how to use Kconfig to add new configuration options to +the kernel and how to use those options in the code. + +### \ No newline at end of file diff --git a/doc/linux-ref/docs/chapters/01-kernel/03-command-line.md b/doc/linux-ref/docs/chapters/01-kernel/03-command-line.md new file mode 100644 index 0000000..bdd5915 --- /dev/null +++ b/doc/linux-ref/docs/chapters/01-kernel/03-command-line.md @@ -0,0 +1,101 @@ +# The Linux Command Line + +The Linux command line is a way to reconfigure parts of the kernel without +having to fully recompile it. + +## Editing the command line +sudo virsh net-start default +You can view your current command line with the following command: + +```bash +cat /proc/cmdline +``` + +*/proc/cmdline* + +``` +[root@arch blix]# cat /proc/cmdline +BOOT_IMAGE=/vmlinuz-linux root=UUID=2d53026e-e884-4578-89eb-0766c74f6e71 +rw zswap.enabled=0 rootfstype=ext4 loglevel=3 quiet +[root@arch blix]# +``` + +The command line is made up of a series of parameters that are separated by a +space. Some parameter have a value, and some are just flags. + +### Updating the command line in GRUB + +*In this manual we wont go over every single bootloader but for more information +you can look at this page +[on the ArchWiki about editing the command line](https://wiki.archlinux.org/title/Kernel_parameters)* + +#### Non-persistent changes + +When in the GRUB menu, hover over the entry you want to edit and press `e` to +edit it. This will bring up a simple text editor where you can edit the GRUB +configuration. + +Look for the line that starts with `linux`, it will be quite long and you can +press `end` on the `linux` line to jump to the end of it. Lets add the `splash` +parameter to the end of the line. To boot this configuration press `ctrl + x` or `F10`. + +You should now either see a blank screen where you would normally see the boot +messages, or you should see a splash screen if your distribution has one. + +Now cat out the command line again and you should see that the `splash` parameter +has been added to the command line. + +#### Making changes persistent + +In your booted system use `vim` or `nano` to edit the file `/etc/default/grub`. +Look for the line: + +``` +GRUB_CMDLINE_LINUX_DEFAULT= +``` + +In there you can append or edit kernel configuration parameters. To then install +this configuration use `grub-mkconfig`: + +```bash +sudo grub-mkconfig -o /boot/grub/grub.cfg +``` + +### Updating the command line in systemd-boot + +Editing command line parameters in systemd-boot is similar to GRUB but you will +need to edit the entry in the `/boot` folder. + +#### Non-persistent changes + +When in the systemd-boot menu, hover over the entry you want to edit and +press `e` to edit it. + +#### Making changes persistent + +Edit the file `/boot/loader/entries/arch.conf` and add the parameters you want +to the end of the `options` line. Then save the file and reboot. + +## Hijacking the command line + +Without using your bootloader, you can force the command line to be something +else. This is done by forcing the `/proc/cmdline` file to use a mounted file instead of the real command line. + +``` +cp /proc/cmdline /tmp/cmdline +# Then edit it as desired +mount --bind /tmp/cmdline /proc/cmdline +``` + +**Note**: This will only change the command line for the user space processes, +the kernel will still use the original command line that is in memory. + +## Kernel boot parameters + +Below is a short list of boot related parameters as a quick guide: + +- `init`: This parameter specifies the initial program that the kernel should +run after fully booting. By default, this is usually set to `/sbin/init`. +- `initrd`: This parameter specifies the initial RAM disk (initrd) image that +the kernel should use during the boot process. The initrd is a temporary file +system that is loaded into memory and used by the kernel during the early stages of booting. diff --git a/doc/linux-ref/docs/chapters/01-kernel/10-boot/01-bootloaders.md b/doc/linux-ref/docs/chapters/01-kernel/10-boot/01-bootloaders.md index 62dae52..04c271f 100644 --- a/doc/linux-ref/docs/chapters/01-kernel/10-boot/01-bootloaders.md +++ b/doc/linux-ref/docs/chapters/01-kernel/10-boot/01-bootloaders.md @@ -35,3 +35,34 @@ along with their support: | Unified kernel image | No | Yes | Yes | Yes | Firmware | +## What a bootloader does + +A bootloader is responsible for: + +1. Locating a kernel image on disk or other media +2. Loading the kernel into memory +3. Loading an initramfs (if present) +4. Providing kernel command line arguments +5. Transferring control to the kernel entry point + +On modern systems, the bootloader may also: + +- Provide a menu for selecting kernel versions +- Handle multiple operating systems +- Load additional modules or microcode updates + +## BIOS vs UEFI boot + +### BIOS (Legacy) + +- Uses MBR (Master Boot Record) +- Bootloader resides in the first sectors of the disk +- Limited space → multi-stage bootloaders (e.g. GRUB stage1/stage2) +- 16-bit real mode at boot + +### UEFI + +- Uses EFI System Partition (ESP) +- Bootloaders are regular `.efi` executables +- Firmware can directly load the kernel (EFI stub) +- No strict size limitations like MBR \ No newline at end of file diff --git a/doc/linux-ref/docs/chapters/02-filesystems/01-filesystem-hierarchy.md b/doc/linux-ref/docs/chapters/02-filesystems/01-filesystem-hierarchy.md index 66c967e..945d337 100644 --- a/doc/linux-ref/docs/chapters/02-filesystems/01-filesystem-hierarchy.md +++ b/doc/linux-ref/docs/chapters/02-filesystems/01-filesystem-hierarchy.md @@ -5,4 +5,256 @@ defines the organization of files and directories on a Linux system. It is based on the Filesystem Hierarchy Standard (FHS) and is used by most Linux distributions. -## \ No newline at end of file +## Standard Hierarchy + +According to the Free Desktop Organization, the standard Linux filesystem +hierarchy is as follows: + +- `/bin`: Essential user command binaries +- `/boot`: Files for the boot loader and kernel +- `/dev`: Device files +- `/etc`: Host-specific system configuration +- `/home`: User home directories +- `/lib`: Shared libraries and kernel modules +- `/media`: Mount points for removable media (CD-ROM, Floppy, etc) +- `/mnt`: Mount point for mounting a filesystem temporarily +- `/opt`: Add-on application software packages +- `/proc`: Virtual filesystem providing process and kernel information as files +- `/root`: Home directory for the root user +- `/run`: Runtime variable data +- `/sbin`: System binaries +- `/srv`: Data for services provided by the system +- `/sys`: Virtual filesystem providing system information and configuration +- `/tmp`: Temporary files +- `/usr`: Secondary hierarchy for read-only user data +- `/var`: Variable data files + +Each of these directories has a specific purpose and contains specific types of +files. + +### /bin + +The `/bin` directory contains executable binaries that are essential for the +system and can be ran by all users. In here you will find common commands such +as `ls`, `cp`, `mv`, `rm`, and many more. + +### /boot + +The `/boot` directory is where the kernel and its related files are stored. This +directory is critical for the boot process, as it contains the kernel image and +the initial RAM filesystem (initramfs) that the kernel uses during boot. + +If you open this directory, you should see three important files: + +- vmlinuz-<version>: This is the compressed Linux kernel image. +- initramfs-<version>.img: This is the initial RAM filesystem image. +- initrd.img-<version>: This is the initial RAM disk image. This is a legacy format that is + mostly unused now, but some older systems may still use it. + +Here is what my `/boot` directory looks like: + +``` +[root@arch boot]# ls -l +total 24916 +drwxr-xr-x 6 root root 4096 Mar 19 19:50 grub +-rwxr-xr-x 1 root root 9038759 Mar 19 19:50 initramfs-linux.img +-rwxr-xr-x 1 root root 16466432 Mar 19 19:50 vmlinuz-linux +11 +[root@arch boot]# +``` + +#### Vmlinuz + +Virtual Memory LINUx gZip. This is the compressed Linux kernel image. When the system boots, the +bootloader loads this image into memory and decompresses it to start the operating system. + +#### RAM Disks + +A RAM disk is a virtual disk that resides in the system’s RAM. It is used during the boot process to +provide a temporary root filesystem before the actual root filesystem is mounted. This allows the kernel +to load necessary drivers and modules. This is because things like controller drivers for disks and other +devices may not work without the RAM disk. + +##### Types of RAM Disks + +- `initramfs`: This is the modern format for the initial RAM filesystem. It is a compressed CPIO +archive that is loaded into memory and used as the initial root filesystem during boot. +- `initrd`: This is the older format for the initial RAM disk. + +Linux supports these compressions on the kernel and initramfs images: + +- `gzip` +- `bzip2` +- `xz` +- `lzma` +- `lzop` +- `lz4` +- `zstd` + + +**Note**: The kernel can not load the initramfs if it is compressed with a compression algorithm that the +kernel has not been compiled with support for. Ensure you enable support for the compression algorithm +using `menuconfig` + +### /dev + +The `/dev` directory contains device files that represent hardware devices and +virtual devices on the system. These files allow user-space programs to interact +with hardware and system resources without needing to interact with the hardware +directly. This increases security and stability since you can give file +permissions to these device files and control who can access them. + +#### A brief overview of device files + +*In this list, `*` denotes a wildcard matching any characters.* + +##### Disks + +- `sd*`: + SCSI disk devices; now used generically for most block storage (SATA, USB). +- `hd*`: + Legacy IDE disk devices; largely deprecated in favor of `sd*`. +- `nvme*`: + NVMe solid-state storage devices. + +##### Terminals + +- `tty*`: + Terminal devices (virtual consoles and some serial interfaces). +- `pts/*`: + Pseudo-terminal slave devices used by terminal emulators and SSH sessions. +- `ttyS*`: + Traditional serial port devices. +- `ttyUSB*`: + USB-to-serial converter devices. +- `ttyACM*`: + USB CDC ACM devices (modems, microcontrollers, etc.). +- `ttyAMA*`: + ARM-specific serial devices (commonly seen on embedded platforms). + +##### Network Interfaces + +- `eth*`: + Traditional Ethernet interface names (legacy naming scheme). +- `wlan*`: + Wireless LAN interfaces. +- `lo`: + Loopback interface. +- `en*`: + Predictable interface names (modern systems, e.g. `enp3s0`). + +##### Sound Devices + +- `snd/*`: + ALSA sound subsystem devices. +- `audio*`: + Legacy audio device interfaces. +- `midi*`: + MIDI devices. +- `pcm*`: + PCM audio devices for playback and capture. + +##### USB and Input Devices + +- `usb/*`: + USB device hierarchy. +- `input/*`: + Input devices (keyboard, mouse, controllers). + +##### Misc Devices + +- `lp*`: + Parallel port (printer) devices. + +##### Non-hardware Devices + +- `null`: + Discards all data written; reads return EOF. +- `zero`: + Produces an infinite stream of null bytes. +- `random`: + Blocking source of random data (entropy-based). +- `urandom`: + Non-blocking random data source. +- `full`: + Always fails writes with ENOSPC ("disk full"). +- `console`: + System console for kernel and early userspace output. + +### /etc + +The `/etc` directory contains host-specific system configuration files. Many of +these are not related to the kernel and more used by user-space programs. + +Privileged users can usually only edit these files + +### /home + +The `/home` directory contains the home directories for regular users. Folders +are owned by the user and have permissions set to allow the user to read, write, and execute files within their home directory. + +User configuration files are stored in the directory with a dot prefix, such as `.bashrc` and are also stored in the `.config` and `.local` directories. These files and directories are hidden by default. + +### /lib + +The `/lib` directory contains essential shared libraries and kernel modules. These libraries are required for the system to boot and run basic commands. + +### /media + +The `/media` directory is used for mounting removable media: + +- `/media/floppy`: Mount point for floppy disks. +- `/media/cdrom`: Mount point for CD-ROMs. +- `/media/cdrecorder`: Mount point for CD writers. +- `/media/zip`: Mount point for ZIP drives. + +You may also see: + +- `/media/usb`: Mount point for USB drives. +- `/media/dvd`: Mount point for DVDs. +- `/media/iso`: Mount point for ISO images. + +But these are not standardized and may not be present on all systems. + +### /mnt + +The `/mnt` directory is a temporary mount point for mounting filesystems. It +must not be used for permanent mounts. It is typically used by system administrators when they need to mount a filesystem temporarily for maintenance. + +### /opt + +Optional application software packages are stored in the `/opt` directory. This is where third-party applications that are not part of the standard distribution can be installed. + +### /proc + +The `/proc` directory is a virtual filesystem that provides process and kernel information as files. It is used to access information about the system and running processes and is managed by the kernel. + +### /root + +The `/root` directory is the home directory for the root user. It is separate from the `/home` directory to ensure that the root user's files are kept separate from regular users' files. + +### /var + +The `/var` directory contains variable data files. This includes log files, spool files, and temporary files that are expected to grow in size over time. + +### /usr + +The `/usr` directory is a secondary hierarchy for read-only user data. It contains the majority of user utilities and applications. This includes subdirectories such as: + +- `/usr/bin`: Non-essential user command binaries. +- `/usr/lib`: Non-essential shared libraries and kernel modules. +- `/usr/local`: Local hierarchy for system administrator-installed software. +- `/usr/share`: Architecture-independent data (e.g., documentation, icons). +- `/usr/include`: Header files for development. +- `/usr/src`: Source code for the kernel and other software. (Not always present) +- `/usr/games`: Game binaries (not always present) + +### /sbin + +Administrative system binaries are stored in the `/sbin` directory. Similar to +`/bin`, but these are intended for use by the system administrator and system +itself. + +### /srv + +The `/srv` directory contains data for services provided by the system. This can include web server files, FTP server files, and other service-related data. \ No newline at end of file diff --git a/doc/linux-ref/docs/chapters/99-reference/99-abbr.md b/doc/linux-ref/docs/chapters/99-reference/99-abbr.md new file mode 100644 index 0000000..45fc9e9 --- /dev/null +++ b/doc/linux-ref/docs/chapters/99-reference/99-abbr.md @@ -0,0 +1,153 @@ +# Abbreviations and Acronyms + +## General + +| Abbrev | Meaning | +|--------|----------------------------------------------| +| ABI | Application Binary Interface | +| API | Application Programming Interface | +| ASCII | American Standard Code for Information | +| | Interchange | +| BIOS | Basic Input/Output System | +| CLI | Command Line Interface | +| CPU | Central Processing Unit | +| DMA | Direct Memory Access | +| ELF | Executable and Linkable Format | +| FHS | Filesystem Hierarchy Standard | +| FS | Filesystem | +| FOSS | Free and Open Source Software | +| GNU | GNU's Not Unix | +| GPU | Graphics Processing Unit | +| GUI | Graphical User Interface | +| I/O | Input/Output | +| IPC | Inter-Process Communication | +| ISO | International Organization for | +| | Standardization | +| KISS | Keep It Simple, Stupid | +| LTS | Long-Term Support | +| MMU | Memory Management Unit | +| OS | Operating System | +| POSIX | Portable Operating System Interface | +| RAM | Random Access Memory | +| ROM | Read-Only Memory | +| SMP | Symmetric Multiprocessing | +| TTY | Teletypewriter | +| UEFI | Unified Extensible Firmware Interface | +| UI | User Interface | +| UUID | Universally Unique Identifier | +| VM | Virtual Machine | +| VMM | Virtual Machine Monitor | + +## Kernel and Low-Level + +| Abbrev | Meaning | +|--------|----------------------------------------------| +| CFS | Completely Fair Scheduler | +| GFP | Get Free Page (allocation flags) | +| HRT | High Resolution Timer | +| IRQ | Interrupt Request | +| ISR | Interrupt Service Routine | +| KASLR | Kernel Address Space Layout Randomization | +| Kbuild | Kernel build system | +| Kconfig| Kernel configuration system | +| LKM | Loadable Kernel Module | +| MM | Memory Management subsystem | +| NMI | Non-Maskable Interrupt | +| NUMA | Non-Uniform Memory Access | +| OOM | Out Of Memory | +| RCU | Read-Copy Update | +| RT | Real-Time | +| TLB | Translation Lookaside Buffer | +| VFS | Virtual Filesystem | + +## Filesystems and Storage + +| Abbrev | Meaning | +|--------|----------------------------------------------| +| ACL | Access Control List | +| Btrfs | B-Tree Filesystem | +| EXT4 | Fourth Extended Filesystem | +| FAT | File Allocation Table | +| GPT | GUID Partition Table | +| MBR | Master Boot Record | +| NFS | Network File System | +| RAID | Redundant Array of Independent Disks | +| tmpfs | Temporary Filesystem | + +## Programming and Toolchain + +| Abbrev | Meaning | +|--------|----------------------------------------------| +| ASM | Assembly Language | +| C99 | ISO C Standard (1999) | +| GCC | GNU Compiler Collection | +| GDB | GNU Debugger | +| GLIBC | GNU C Library | +| LD | Linker | +| LIBC | Standard C Library | +| OBJ | Object File | +| PLT | Procedure Linkage Table | +| GOT | Global Offset Table | +| REPL | Read-Eval-Print Loop | + +## Userspace and System + +| Abbrev | Meaning | +|--------|----------------------------------------------| +| DBus | Desktop Bus (IPC system) | +| ENV | Environment Variable | +| INIT | Initial process (PID 1) | +| PAM | Pluggable Authentication Modules | +| PID | Process Identifier | +| PPID | Parent Process Identifier | +| PROCFS | Process Filesystem (/proc) | +| SYSFS | System Filesystem (/sys) | +| UID | User Identifier | +| GID | Group Identifier | + +## Shell and Scripting + +| Abbrev | Meaning | +|--------|----------------------------------------------| +| BASH | Bourne Again SHell | +| FD | File Descriptor | +| PWD | Present Working Directory | +| SH | Bourne Shell | +| STDIN | Standard Input | +| STDOUT | Standard Output | +| STDERR | Standard Error | + +## Networking + +| Abbrev | Meaning | +|--------|----------------------------------------------| +| ARP | Address Resolution Protocol | +| DNS | Domain Name System | +| HTTP | Hypertext Transfer Protocol | +| HTTPS | Hypertext Transfer Protocol Secure | +| ICMP | Internet Control Message Protocol | +| IP | Internet Protocol | +| IPv4 | Internet Protocol Version 4 | +| IPv6 | Internet Protocol Version 6 | +| MAC | Media Access Control (address) | +| NAT | Network Address Translation | +| OSI | Open Systems Interconnection | +| TCP | Transmission Control Protocol | +| UDP | User Datagram Protocol | +| UNIX | Unix Domain Socket | + +## Hardware + +| Abbrev | Meaning | +|--------|----------------------------------------------| +| ACPI | Advanced Configuration and Power Interface | +| APIC | Advanced Programmable Interrupt Controller | +| ATA | Advanced Technology Attachment | +| CMOS | Complementary Metal-Oxide Semiconductor | +| I2C | Inter-Integrated Circuit | +| PCI | Peripheral Component Interconnect | +| PCIe | Peripheral Component Interconnect Express | +| PS/2 | Personal System/2 | +| SATA | Serial ATA | +| USB | Universal Serial Bus | +| VGA | Video Graphics Array | diff --git a/doc/linux-ref/docs/chapters/99-sources.md b/doc/linux-ref/docs/chapters/99-sources.md index 10b1dab..05ca7fb 100644 --- a/doc/linux-ref/docs/chapters/99-sources.md +++ b/doc/linux-ref/docs/chapters/99-sources.md @@ -18,4 +18,7 @@ Many thanks to all the sources that were used to write this book: 1. https://wiki.archlinux.org/title/Kernel 2. https://static.sched.com/hosted_files/osseu2020/f5/2020_ELCE_initcalls_myjosserand.pdf 4. https://www.linuxfromscratch.org/blfs/view/svn/postlfs/initramfs.html -3. https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard \ No newline at end of file +3. https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard +4. https://elixir.bootlin.com/linux/v6.19.8/source/include/linux/init.h#L45 +5. https://www.kernel.org/doc/html/v4.14/admin-guide/kernel-parameters.html +6. https://wiki.archlinux.org/title/Kernel_parameters \ No newline at end of file[COMMIT END](C) 2025 0x4248 (C) 2025 4248 Media and 4248 Systems, All part of 0x4248 See LICENCE files for more information. Not all files are by 0x4248 always check Licencing.