The User/Kernel Boundary

Every concept in this section is presented in six parts: the problem it solves, where it lives in the kernel, who owns or interacts with it, the structures, syscalls, and code paths involved, an experiment, and the failure mode when you get it wrong.

This chapter covers three concepts: the syscall entry path, user pointers, and the full uapi surface — because the syscall is only the most obvious of the ways user space reaches you.

Concept 1: The Syscall Entry Path

1. What problem it solves

The kernel owns the hardware and the memory of every process. A program needs to ask it for things. The hardware provides exactly one primitive for that: an instruction that raises the privilege level and jumps to a fixed address the kernel chose in advance.

Everything else — the syscall number, the argument registers, the table, the return convention — is a software contract built on top of that one instruction, and it is the most permanent contract in the system.

2. Where it exists in the kernel

arch/, and then generic C. The split is exactly the interesting part:

 ┌──────────────────────────────────────────────────────────────────────────┐
 │  arch/<arch>/entry/          ARCHITECTURE-SPECIFIC, mostly assembly      │
 │    • switch to the kernel stack (never trust the user stack)             │
 │    • save every user register into `struct pt_regs`                      │
 │    • KPTI: switch page tables (the Meltdown mitigation)                  │
 │    • speculation barriers and mitigations                                │
 │    • bounds-check the syscall number, index the table, call              │
 └──────────────────────────────────────────────────────────────────────────┘
                                    │
 ┌──────────────────────────────────▼───────────────────────────────────────┐
 │  fs/  mm/  kernel/  net/ …       GENERIC C, one implementation           │
 │    SYSCALL_DEFINE3(read, unsigned int, fd, char __user *, buf, size_t, n)│
 │    — the same function body serves every architecture                    │
 └──────────────────────────────────────────────────────────────────────────┘

Find it on your tree:

ls arch/x86/entry/            arch/arm64/kernel/entry.S
rg -n "do_syscall_64"         arch/x86/entry/common.c
rg -n "el0_svc|invoke_syscall" arch/arm64/kernel/
ls arch/x86/entry/syscalls/   # the .tbl files the table is generated from
rg -n "SYSCALL_DEFINE3\(read" fs/read_write.c

Note: SYSCALL_DEFINEn is a macro, which is why grepping for sys_read finds less than you expect. It expands into several functions: the real body, a wrapper that sign-extends arguments, and an architecture wrapper that unpacks struct pt_regs. Read the macro once so the expansion stops being mysterious: rg -n -A 25 "define __SYSCALL_DEFINEx" include/linux/syscalls.h.

3. Who owns or interacts with it

4. The structures, syscalls, and code paths

The argument registers differ per architecture and you should never hardcode them, but knowing they exist explains a lot:

Note x86-64 uses r10 rather than rcx for the fourth argument: the syscall instruction clobbers rcx with the return address. That is a hardware detail leaking into an ABI, permanently.

5. Experiment

CLAIM. The syscall number, not the name, is the contract — and you can call a syscall with no libc wrapper at all.

METHOD.

cat > /tmp/raw.c <<'EOF'
#define _GNU_SOURCE
#include <unistd.h>
#include <sys/syscall.h>
#include <stdio.h>
int main(void) {
    /* No libc wrapper. Just the number and the arguments. */
    long pid = syscall(SYS_getpid);
    char msg[] = "written by syscall(2) directly\n";
    syscall(SYS_write, 1, msg, sizeof(msg) - 1);
    printf("getpid via syscall(): %ld\n", pid);
    /* An invalid number: what does the kernel do? */
    long r = syscall(4242424, 0, 0, 0);
    printf("bogus syscall returned %ld, errno %d\n", r, errno);
    return 0;
}
EOF
gcc -o /tmp/raw /tmp/raw.c && /tmp/raw
strace -e trace=write,getpid /tmp/raw

PREDICT FIRST: what does an unimplemented syscall number return — a crash, -ENOSYS, or undefined behavior? And does strace show your raw calls differently from libc-issued ones?

Now watch the entry path itself:

sudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter /comm == "raw"/ { @[args.id] = count(); }' &
/tmp/raw; sleep 1; kill %1
# Map the numbers back:
grep -w -E "getpid|write" /usr/include/asm/unistd_64.h 2>/dev/null || \
  ausyscall --dump 2>/dev/null | head -20

6. Failure mode

Concept 2: User Pointers

1. What problem it solves

A syscall argument can be a pointer. That pointer is a number chosen by an untrusted program, and the kernel runs with the privilege to dereference anything.

Three separate problems have to be solved at once, and only one of them is "is the address valid":

  1. IS IT USER MEMORY?     A user program can pass a KERNEL address. Dereferencing
                            it would read or write kernel memory on request — the
                            most direct privilege escalation there is.

  2. IS IT MAPPED?          It may be unmapped, or swapped out, or a not-yet-faulted
                            page of a file mapping. A plain dereference would oops
                            the kernel instead of returning -EFAULT to the caller.

  3. IS IT STILL THERE?     Another thread in the same process can unmap or change
                            it between your two reads. Your validation and your use
                            are not atomic.

The kernel's answer is a small family of accessor functions that solve all three — and a hardware feature on modern CPUs that makes the alternative fail loudly.

2. Where it exists in the kernel

Generic API in include/linux/uaccess.h, architecture implementations under arch/<arch>/include/asm/uaccess.h and arch/<arch>/lib/.

rg -n "copy_from_user|copy_to_user" include/linux/uaccess.h | head
rg -n "define access_ok" arch/x86/include/asm/uaccess.h arch/arm64/include/asm/uaccess.h
find Documentation -name 'exception-tables*'      # how a fault inside copy_*_user is survived

The mechanism that makes copy_from_user able to fault without crashing is the exception table: the copy loop's instructions are registered in a special ELF section, and the page-fault handler consults it. If the faulting instruction is in the table, the kernel jumps to a fixup that returns "bytes not copied" instead of oopsing.

rg -n "fixup_exception" arch/x86/mm/extable.c arch/arm64/mm/extable.c

On modern hardware there is a second layer: SMAP (x86) and PAN (arm64) make any kernel access to a user address fault, unless it is bracketed by the explicit user_access_begin()/user_access_end() that the accessors use internally. This is why a direct dereference of a user pointer is a hard, immediate failure on a current machine rather than a silent, occasionally-working bug.

grep -o -w -E 'smap|pan' /proc/cpuinfo | sort -u    # x86: 'smap' in flags

3. Who owns or interacts with it

4. The structures, syscalls, and code paths

Warning: copy_from_user returning nonzero is not an error code. It is a count. The correct idiom is

if (copy_from_user(&karg, uarg, sizeof(karg)))
        return -EFAULT;

Writing ret = copy_from_user(...); if (ret) return ret; returns a positive byte count to user space as if it were a success value. This bug has shipped. Repeatedly.

The two patterns that matter most.

Copying in — validate the kernel copy, never the user memory:

struct my_arg karg;

/* One copy. Everything after this reads the KERNEL copy, which cannot
 * change under us. Validating uarg and then re-reading it would be a
 * classic double-fetch (TOCTOU) bug.                                    */
if (copy_from_user(&karg, uarg, sizeof(karg)))
        return -EFAULT;

if (karg.flags & ~MY_VALID_FLAGS)     /* reject unknown bits: this is what
        return -EINVAL;                * keeps the flags field extensible */
if (karg.len > MY_MAX_LEN)
        return -EINVAL;

Copying out — never leak what you did not intend to send:

struct my_result kres;

/* memset the WHOLE struct, not just the fields. The compiler is not
 * required to initialize padding bytes, and padding copied to user space
 * is an information leak of whatever was on the kernel stack.           */
memset(&kres, 0, sizeof(kres));
kres.a = 1;
kres.b = 2;

if (copy_to_user(ures, &kres, sizeof(kres)))
        return -EFAULT;

And one more, for anything that indexes an array with a user-controlled value:

if (idx >= ARRAY_SIZE(table))
        return -EINVAL;
/* The bounds check above can be speculated PAST (Spectre v1). This masks
 * the index so a mis-speculated path cannot read out of bounds.          */
idx = array_index_nospec(idx, ARRAY_SIZE(table));
return table[idx];

5. Experiment

CLAIM. On modern hardware, dereferencing a user pointer from kernel code fails immediately and loudly — and the failure is the same for a valid user address as for an invalid one, which is exactly what SMAP/PAN was added to guarantee.

METHOD. In a scratch module (do this in the guest, never on your host):

static ssize_t bad_write(struct file *f, const char __user *ubuf,
                         size_t len, loff_t *off)
{
        char c;

        /* WRONG ON PURPOSE. sparse will flag this with make C=1.        */
        c = *(const char *)ubuf;         /* direct dereference           */
        pr_info("read byte 0x%02x\n", c);
        return len;
}

PREDICT FIRST, before you build it, for each case:

Then run all three, and also run make C=1 M=$PWD and see what sparse says before you ever load it. Restore the copy_from_user version and confirm all three now behave correctly (-EFAULT for the bad ones).

RESULT. Record what actually happened for each case. On a machine with SMAP/PAN the first case fails too — and understanding why that is the desired outcome is the point of the experiment.

6. Failure mode

Note: That second-to-last row surprises people. copy_to_user() can sleep, because the destination page may not be resident and faulting it in may require I/O. It therefore belongs in the list of "functions that sleep and do not look like it" from the mental model.

Concept 3: The Full uapi Surface

1. What problem it solves

"The kernel's interface to user space" is much larger than the syscall table, and every part of it is subject to the rule: if a program worked before your patch and does not work after it, the kernel is wrong.

You need to know which surfaces are frozen, which are explicitly not, and where the line is — because the most common way a first patch gets rejected on the merits is "that changes an interface someone depends on."

2. Where it exists in the kernel

ls include/uapi/linux/ | head -20        # the headers user space compiles against
ls Documentation/ABI/                    # stable/ testing/ obsolete/ removed/
ls Documentation/ABI/stable/             # things you may not change
$EDITOR Documentation/ABI/README

Documentation/ABI/ is the part people do not know exists. It is the register of promises, split by how binding each one is, and adding an entry to it is part of adding a sysfs attribute.

3. Who owns or interacts with it

Tip: When you want to expose a new number to user space, ask in this order: Can it be a debugfs file? (No stability promise, no argument, ships today.) Then: a sysfs attribute with an ABI/testing entry? Then: a tracepoint? Then: a netlink attribute? A new syscall is the last resort and needs a case. This ordering is most of what Lab 5 is about.

4. The structures and code paths

The four things you touch when adding a user-visible interface:

  include/uapi/linux/<thing>.h    the struct, the constants, the ioctl numbers
                                  → must compile standalone in user space
                                  → __u32 / __u64, never u32 / u64
                                  → explicit padding, no implicit holes
                                  → SPDX: GPL-2.0 WITH Linux-syscall-note

  Documentation/ABI/testing/…     one entry per attribute: What, Date,
                                  KernelVersion, Contact, Description

  Documentation/userspace-api/    prose for anything non-obvious
    ioctl/ioctl-number.rst        the registry, if you add an ioctl

  tools/testing/selftests/…       the test that proves the interface works
                                  AND documents what you promised

The uapi struct rules, which reviewers check every time:

/* include/uapi/linux/mylab.h */
/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
#ifndef _UAPI_LINUX_MYLAB_H
#define _UAPI_LINUX_MYLAB_H

#include <linux/types.h>          /* __u32 and friends — NOT <linux/kernel.h> */

struct mylab_config {
        __u32 flags;              /* fixed width, uapi spelling             */
        __u32 count;              /* naturally aligned                      */
        __u64 addr;               /* __u64 even for pointers: same size on
                                   * 32- and 64-bit userspace               */
        __u32 timeout_ms;
        __u32 __reserved;         /* EXPLICIT padding, zeroed and checked,
                                   * so the hole cannot leak and can later
                                   * become a real field                    */
};

#define MYLAB_FLAG_A    (1U << 0)
#define MYLAB_FLAG_B    (1U << 1)
#define MYLAB_FLAG_ALL  (MYLAB_FLAG_A | MYLAB_FLAG_B)

#endif /* _UAPI_LINUX_MYLAB_H */

Every line of that has a reason:

5. Experiment

CLAIM. A struct with an implicit padding hole has a different layout under different ABIs, and the kernel cannot silently paper over it.

METHOD.

cat > /tmp/pad.c <<'EOF'
#include <stdio.h>
#include <stddef.h>
struct bad  { unsigned int a; unsigned long long b; unsigned int c; };
struct good { unsigned int a; unsigned int c; unsigned long long b; };
int main(void) {
    printf("bad : size=%zu  a@%zu b@%zu c@%zu\n", sizeof(struct bad),
           offsetof(struct bad,a), offsetof(struct bad,b), offsetof(struct bad,c));
    printf("good: size=%zu  a@%zu c@%zu b@%zu\n", sizeof(struct good),
           offsetof(struct good,a), offsetof(struct good,c), offsetof(struct good,b));
    return 0;
}
EOF
gcc      -o /tmp/pad64 /tmp/pad.c && /tmp/pad64
gcc -m32 -o /tmp/pad32 /tmp/pad.c 2>/dev/null && /tmp/pad32 || \
  echo "(no 32-bit libc; the 64-bit output alone still shows the hole)"

PREDICT FIRST: how many bytes is struct bad? How many of them are padding? Where are they?

Then find the tool the kernel uses for this, which reads your built kernel's debug info:

cd ~/kernel/build
pahole -C task_struct vmlinux | head -40          # every hole, annotated
pahole --help | grep -i hole

pahole printing /* XXX 4 bytes hole, try to pack */ is the review comment you want to have already fixed.

6. Failure mode

Putting It Together: What a Reviewer Checks

When your patch touches this boundary, this is the list running in the reviewer's head. Run it yourself first.

  [ ] Every user pointer is `__user`, and make C=1 is clean
  [ ] Every copy_from_user / copy_to_user return value is checked
  [ ] Nothing is validated in user memory and then re-read (no double fetch)
  [ ] Structs copied out are memset to zero first
  [ ] Every array index from user space is bounds-checked, and masked if hot
  [ ] Unknown flag bits are rejected with -EINVAL
  [ ] uapi structs have explicit padding and fixed-width types, verified with pahole
  [ ] The compat/32-bit case is handled or explicitly documented as unsupported
  [ ] Nothing that can fault is done while holding a spinlock
  [ ] The new interface is on the least-permanent surface that will do the job
  [ ] There is a selftest, and it is also the documentation of the promise

Validation / Self-check

1. Name every step between a user program executing syscall and its handler body running. Which are architecture-specific and which are generic?
2. Why does x86-64 pass the fourth syscall argument in r10 instead of rcx?
3. What does copy_from_user return, and write the correct idiom for checking it from memory.
4. Name three distinct problems the user-pointer accessors solve that a range check alone does not.
5. What is the exception table, and what would happen without it when a user page is not resident?
6. What are SMAP and PAN, and how do they change the failure mode of a direct dereference?
7. Describe a double-fetch bug concretely, and give the structural fix (not "be careful").
8. Why must a struct be memset before copy_to_user, when every field is assigned?
9. Why is __u64 used for pointer-valued fields in uapi structs?
10. Rank these by how permanent the promise is: a debugfs file, a sysfs attribute, a tracepoint, an ioctl, a syscall. Where would you put a new diagnostic counter, and why?
11. You want to add a field to an existing uapi struct. What are the only two safe ways?
12. Your ioctl handler calls copy_to_user while holding a spinlock it needs for consistency. The debug kernel BUGs. What is the fix — and what is wrong with the obvious one of switching to GFP_ATOMIC?

Next: Kernel C — the dialect, and every rule in it that will bite you.