admin/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2025-12-07 20:06:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge tag 'objtool-core-2025-12-01' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Browse Source 
Pull objtool updates from Ingo Molnar:

 - klp-build livepatch module generation (Josh Poimboeuf)

   Introduce new objtool features and a klp-build script to generate
   livepatch modules using a source .patch as input.

   This builds on concepts from the longstanding out-of-tree kpatch
   project which began in 2012 and has been used for many years to
   generate livepatch modules for production kernels. However, this is a
   complete rewrite which incorporates hard-earned lessons from 12+
   years of maintaining kpatch.

   Key improvements compared to kpatch-build:

    - Integrated with objtool: Leverages objtool's existing control-flow
      graph analysis to help detect changed functions.

    - Works on vmlinux.o: Supports late-linked objects, making it
      compatible with LTO, IBT, and similar.

    - Simplified code base: ~3k fewer lines of code.

    - Upstream: No more out-of-tree #ifdef hacks, far less cruft.

    - Cleaner internals: Vastly simplified logic for
      symbol/section/reloc inclusion and special section extraction.

    - Robust __LINE__ macro handling: Avoids false positive binary diffs
      caused by the __LINE__ macro by introducing a fix-patch-lines
      script which injects #line directives into the source .patch to
      preserve the original line numbers at compile time.

 - Disassemble code with libopcodes instead of running objdump
   (Alexandre Chartre)

 - Disassemble support (-d option to objtool) by Alexandre Chartre,
   which supports the decoding of various Linux kernel code generation
   specials such as alternatives:

      17ef:  sched_balance_find_dst_group+0x62f                 mov    0x34(%r9),%edx
      17f3:  sched_balance_find_dst_group+0x633               | <alternative.17f3>             | X86_FEATURE_POPCNT
      17f3:  sched_balance_find_dst_group+0x633               | call   0x17f8 <__sw_hweight64> | popcnt %rdi,%rax
      17f8:  sched_balance_find_dst_group+0x638                 cmp    %eax,%edx

   ... jump table alternatives:

      1895:  sched_use_asym_prio+0x5                            test   $0x8,%ch
      1898:  sched_use_asym_prio+0x8                            je     0x18a9 <sched_use_asym_prio+0x19>
      189a:  sched_use_asym_prio+0xa                          | <jump_table.189a>                        | JUMP
      189a:  sched_use_asym_prio+0xa                          | jmp    0x18ae <sched_use_asym_prio+0x1e> | nop2
      189c:  sched_use_asym_prio+0xc                            mov    $0x1,%eax
      18a1:  sched_use_asym_prio+0x11                           and    $0x80,%ecx

   ... exception table alternatives:

    native_read_msr:
      5b80:  native_read_msr+0x0                                                     mov    %edi,%ecx
      5b82:  native_read_msr+0x2                                                   | <ex_table.5b82> | EXCEPTION
      5b82:  native_read_msr+0x2                                                   | rdmsr           | resume at 0x5b84 <native_read_msr+0x4>
      5b84:  native_read_msr+0x4                                                     shl    $0x20,%rdx

   .... x86 feature flag decoding (also see the X86_FEATURE_POPCNT
        example in sched_balance_find_dst_group() above):

      2faaf:  start_thread_common.constprop.0+0x1f                                    jne    0x2fba4 <start_thread_common.constprop.0+0x114>
      2fab5:  start_thread_common.constprop.0+0x25                                  | <alternative.2fab5>                  | X86_FEATURE_ALWAYS                                  | X86_BUG_NULL_SEG
      2fab5:  start_thread_common.constprop.0+0x25                                  | jmp    0x2faba <.altinstr_aux+0x2f4> | jmp    0x4b0 <start_thread_common.constprop.0+0x3f> | nop5
      2faba:  start_thread_common.constprop.0+0x2a                                    mov    $0x2b,%eax

   ... NOP sequence shortening:

      1048e2:  snapshot_write_finalize+0xc2                                            je     0x104917 <snapshot_write_finalize+0xf7>
      1048e4:  snapshot_write_finalize+0xc4                                            nop6
      1048ea:  snapshot_write_finalize+0xca                                            nop11
      1048f5:  snapshot_write_finalize+0xd5                                            nop11
      104900:  snapshot_write_finalize+0xe0                                            mov    %rax,%rcx
      104903:  snapshot_write_finalize+0xe3                                            mov    0x10(%rdx),%rax

   ... and much more.

 - Function validation tracing support (Alexandre Chartre)

 - Various -ffunction-sections fixes (Josh Poimboeuf)

 - Clang AutoFDO (Automated Feedback-Directed Optimizations) support
   (Josh Poimboeuf)

 - Misc fixes and cleanups (Borislav Petkov, Chen Ni, Dylan Hatch, Ingo
   Molnar, John Wang, Josh Poimboeuf, Pankaj Raghav, Peter Zijlstra,
   Thorsten Blum)

* tag 'objtool-core-2025-12-01' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (129 commits)
  objtool: Fix segfault on unknown alternatives
  objtool: Build with disassembly can fail when including bdf.h
  objtool: Trim trailing NOPs in alternative
  objtool: Add wide output for disassembly
  objtool: Compact output for alternatives with one instruction
  objtool: Improve naming of group alternatives
  objtool: Add Function to get the name of a CPU feature
  objtool: Provide access to feature and flags of group alternatives
  objtool: Fix address references in alternatives
  objtool: Disassemble jump table alternatives
  objtool: Disassemble exception table alternatives
  objtool: Print addresses with alternative instructions
  objtool: Disassemble group alternatives
  objtool: Print headers for alternatives
  objtool: Preserve alternatives order
  objtool: Add the --disas=<function-pattern> action
  objtool: Do not validate IBT for .return_sites and .call_sites
  objtool: Improve tracing of alternative instructions
  objtool: Add functions to better name alternatives
  objtool: Identify the different types of alternatives
  ...
This commit is contained in:
Linus Torvalds
2025-12-01 20:18:59 -08:00
parent b53440f8e5 6ec33db1aa
commit 63e6995005
99 changed files with 7642 additions and 1296 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
MAINTAINERS
View File

@@ -14459,10 +14459,11 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/livepatching/livepatching.g
F: Documentation/ABI/testing/sysfs-kernel-livepatch
F: Documentation/livepatch/
F: arch/powerpc/include/asm/livepatch.h
F: include/linux/livepatch.h
F: include/linux/livepatch*.h
F: kernel/livepatch/
F: kernel/module/livepatch.c
F: samples/livepatch/
F: scripts/livepatch/
F: tools/testing/selftests/livepatch/
LLC (802.2)

2
arch/s390/include/asm/nospec-insn.h
View File

@@ -19,7 +19,7 @@
#ifdef CONFIG_EXPOLINE_EXTERN
SYM_CODE_START(\name)
#else
.pushsection .text.\name,"axG",@progbits,\name,comdat
.pushsection .text..\name,"axG",@progbits,\name,comdat
.globl \name
.hidden \name
.type \name,@function

2
arch/s390/kernel/vmlinux.lds.S
View File

@@ -51,7 +51,7 @@ SECTIONS
IRQENTRY_TEXT
SOFTIRQENTRY_TEXT
FTRACE_HOTPATCH_TRAMPOLINES_TEXT
*(.text.*_indirect_*)
*(.text..*_indirect_*)
*(.gnu.warning)
. = ALIGN(PAGE_SIZE);
_etext = .; /* End of text section */

1
arch/x86/Kconfig
View File

@@ -261,6 +261,7 @@ config X86
select HAVE_FUNCTION_ERROR_INJECTION
select HAVE_KRETPROBES
select HAVE_RETHOOK
select HAVE_KLP_BUILD if X86_64
select HAVE_LIVEPATCH if X86_64
select HAVE_MIXED_BREAKPOINTS_REGS
select HAVE_MOD_ARCH_SPECIFIC

2
arch/x86/boot/startup/Makefile
View File

@@ -36,7 +36,7 @@ $(patsubst %.o,$(obj)/%.o,$(lib-y)): OBJECT_FILES_NON_STANDARD := y
# relocations, even if other objtool actions are being deferred.
#
$(pi-objs): objtool-enabled = 1
$(pi-objs): objtool-args = $(if $(delay-objtool),,$(objtool-args-y)) --noabs
$(pi-objs): objtool-args = $(if $(delay-objtool),--dry-run,$(objtool-args-y)) --noabs
#
# Confine the startup code by prefixing all symbols with __pi_ (for position

4
arch/x86/include/asm/alternative.h
View File

@@ -198,6 +198,7 @@ static inline int alternatives_text_reserved(void *start, void *end)
#define ALTINSTR_ENTRY(ft_flags) \
".pushsection .altinstructions,\"a\"\n" \
ANNOTATE_DATA_SPECIAL \
" .long 771b - .\n" /* label */ \
" .long 774f - .\n" /* new instruction */ \
" .4byte " __stringify(ft_flags) "\n" /* feature + flags */ \
@@ -207,6 +208,7 @@ static inline int alternatives_text_reserved(void *start, void *end)
#define ALTINSTR_REPLACEMENT(newinstr) /* replacement */ \
".pushsection .altinstr_replacement, \"ax\"\n" \
ANNOTATE_DATA_SPECIAL \
"# ALT: replacement\n" \
"774:\n\t" newinstr "\n775:\n" \
".popsection\n"
@@ -337,6 +339,7 @@ void nop_func(void);
* instruction. See apply_alternatives().
*/
.macro altinstr_entry orig alt ft_flags orig_len alt_len
ANNOTATE_DATA_SPECIAL
.long \orig - .
.long \alt - .
.4byte \ft_flags
@@ -365,6 +368,7 @@ void nop_func(void);
.popsection ; \
.pushsection .altinstr_replacement,"ax" ; \
743: \
ANNOTATE_DATA_SPECIAL ; \
newinst ; \
744: \
.popsection ;

5
arch/x86/include/asm/asm.h
View File

@@ -2,6 +2,8 @@
#ifndef _ASM_X86_ASM_H
#define _ASM_X86_ASM_H
#include <linux/annotate.h>
#ifdef __ASSEMBLER__
# define __ASM_FORM(x, ...) x,## __VA_ARGS__
# define __ASM_FORM_RAW(x, ...) x,## __VA_ARGS__
@@ -132,6 +134,7 @@ static __always_inline __pure void *rip_rel_ptr(void *p)
# define _ASM_EXTABLE_TYPE(from, to, type) \
.pushsection "__ex_table","a" ; \
.balign 4 ; \
ANNOTATE_DATA_SPECIAL ; \
.long (from) - . ; \
.long (to) - . ; \
.long type ; \
@@ -179,6 +182,7 @@ static __always_inline __pure void *rip_rel_ptr(void *p)
# define _ASM_EXTABLE_TYPE(from, to, type) \
" .pushsection \"__ex_table\",\"a\"\n" \
" .balign 4\n" \
ANNOTATE_DATA_SPECIAL \
" .long (" #from ") - .\n" \
" .long (" #to ") - .\n" \
" .long " __stringify(type) " \n" \
@@ -187,6 +191,7 @@ static __always_inline __pure void *rip_rel_ptr(void *p)
# define _ASM_EXTABLE_TYPE_REG(from, to, type, reg) \
" .pushsection \"__ex_table\",\"a\"\n" \
" .balign 4\n" \
ANNOTATE_DATA_SPECIAL \
" .long (" #from ") - .\n" \
" .long (" #to ") - .\n" \
DEFINE_EXTABLE_TYPE_REG \

1
arch/x86/include/asm/bug.h
View File

@@ -57,6 +57,7 @@
#define _BUG_FLAGS_ASM(ins, file, line, flags, size, extra) \
"1:\t" ins "\n" \
".pushsection __bug_table,\"aw\"\n" \
ANNOTATE_DATA_SPECIAL \
__BUG_ENTRY(file, line, flags) \
"\t.org 2b + " size "\n" \
".popsection\n" \

1
arch/x86/include/asm/cpufeature.h
View File

@@ -101,6 +101,7 @@ static __always_inline bool _static_cpu_has(u16 bit)
asm goto(ALTERNATIVE_TERNARY("jmp 6f", %c[feature], "", "jmp %l[t_no]")
".pushsection .altinstr_aux,\"ax\"\n"
"6:\n"
ANNOTATE_DATA_SPECIAL
" testb %[bitnum], %a[cap_byte]\n"
" jnz %l[t_yes]\n"
" jmp %l[t_no]\n"

1
arch/x86/include/asm/jump_label.h
View File

@@ -15,6 +15,7 @@
#define JUMP_TABLE_ENTRY(key, label) \
".pushsection __jump_table, \"aw\" \n\t" \
_ASM_ALIGN "\n\t" \
ANNOTATE_DATA_SPECIAL \
".long 1b - . \n\t" \
".long " label " - . \n\t" \
_ASM_PTR " " key " - . \n\t" \

2
arch/x86/include/asm/smp.h
View File

@@ -109,7 +109,7 @@ int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
int native_kick_ap(unsigned int cpu, struct task_struct *tidle);
int native_cpu_disable(void);
void __noreturn hlt_play_dead(void);
void native_play_dead(void);
void __noreturn native_play_dead(void);
void play_dead_common(void);
void wbinvd_on_cpu(int cpu);
void wbinvd_on_all_cpus(void);

51
arch/x86/kernel/alternative.c
View File

@@ -2244,21 +2244,34 @@ int alternatives_text_reserved(void *start, void *end)
* See entry_{32,64}.S for more details.
*/
/*
* We define the int3_magic() function in assembly to control the calling
* convention such that we can 'call' it from assembly.
*/
extern void int3_magic(unsigned int *ptr); /* defined in asm */
extern void int3_selftest_asm(unsigned int *ptr);
asm (
" .pushsection .init.text, \"ax\", @progbits\n"
" .type int3_magic, @function\n"
"int3_magic:\n"
" .type int3_selftest_asm, @function\n"
"int3_selftest_asm:\n"
ANNOTATE_NOENDBR
" movl $1, (%" _ASM_ARG1 ")\n"
/*
* INT3 padded with NOP to CALL_INSN_SIZE. The INT3 triggers an
* exception, then the int3_exception_nb notifier emulates a call to
* int3_selftest_callee().
*/
" int3; nop; nop; nop; nop\n"
ASM_RET
" .size int3_magic, .-int3_magic\n"
" .size int3_selftest_asm, . - int3_selftest_asm\n"
" .popsection\n"
);
extern void int3_selftest_callee(unsigned int *ptr);
asm (
" .pushsection .init.text, \"ax\", @progbits\n"
" .type int3_selftest_callee, @function\n"
"int3_selftest_callee:\n"
ANNOTATE_NOENDBR
" movl $0x1234, (%" _ASM_ARG1 ")\n"
ASM_RET
" .size int3_selftest_callee, . - int3_selftest_callee\n"
" .popsection\n"
);
@@ -2267,7 +2280,7 @@ extern void int3_selftest_ip(void); /* defined in asm below */
static int __init
int3_exception_notify(struct notifier_block *self, unsigned long val, void *data)
{
unsigned long selftest = (unsigned long)&int3_selftest_ip;
unsigned long selftest = (unsigned long)&int3_selftest_asm;
struct die_args *args = data;
struct pt_regs *regs = args->regs;
@@ -2282,7 +2295,7 @@ int3_exception_notify(struct notifier_block *self, unsigned long val, void *data
if (regs->ip - INT3_INSN_SIZE != selftest)
return NOTIFY_DONE;
int3_emulate_call(regs, (unsigned long)&int3_magic);
int3_emulate_call(regs, (unsigned long)&int3_selftest_callee);
return NOTIFY_STOP;
}
@@ -2298,19 +2311,11 @@ static noinline void __init int3_selftest(void)
BUG_ON(register_die_notifier(&int3_exception_nb));
/*
* Basically: int3_magic(&val); but really complicated :-)
*
* INT3 padded with NOP to CALL_INSN_SIZE. The int3_exception_nb
* notifier above will emulate CALL for us.
* Basically: int3_selftest_callee(&val); but really complicated :-)
*/
asm volatile ("int3_selftest_ip:\n\t"
ANNOTATE_NOENDBR
" int3; nop; nop; nop; nop\n\t"
: ASM_CALL_CONSTRAINT
: __ASM_SEL_RAW(a, D) (&val)
: "memory");
int3_selftest_asm(&val);
BUG_ON(val != 1);
BUG_ON(val != 0x1234);
unregister_die_notifier(&int3_exception_nb);
}

4
arch/x86/kernel/kprobes/opt.c
View File

@@ -103,7 +103,6 @@ static void synthesize_set_arg1(kprobe_opcode_t *addr, unsigned long val)
asm (
".pushsection .rodata\n"
"optprobe_template_func:\n"
".global optprobe_template_entry\n"
"optprobe_template_entry:\n"
#ifdef CONFIG_X86_64
@@ -160,9 +159,6 @@ asm (
"optprobe_template_end:\n"
".popsection\n");
void optprobe_template_func(void);
STACK_FRAME_NON_STANDARD(optprobe_template_func);
#define TMPL_CLAC_IDX \
((long)optprobe_template_clac - (long)optprobe_template_entry)
#define TMPL_MOVE_IDX \

15
arch/x86/kernel/module.c
View File

@@ -97,6 +97,7 @@ static int __write_relocate_add(Elf64_Shdr *sechdrs,
DEBUGP("%s relocate section %u to %u\n",
apply ? "Applying" : "Clearing",
relsec, sechdrs[relsec].sh_info);
for (i = 0; i < sechdrs[relsec].sh_size / sizeof(*rel); i++) {
size_t size;
@@ -162,15 +163,17 @@ static int __write_relocate_add(Elf64_Shdr *sechdrs,
if (apply) {
if (memcmp(loc, &zero, size)) {
pr_err("x86/modules: Invalid relocation target, existing value is nonzero for type %d, loc %p, val %Lx\n",
(int)ELF64_R_TYPE(rel[i].r_info), loc, val);
pr_err("x86/modules: Invalid relocation target, existing value is nonzero for sec %u, idx %u, type %d, loc %lx, val %llx\n",
relsec, i, (int)ELF64_R_TYPE(rel[i].r_info),
(unsigned long)loc, val);
return -ENOEXEC;
}
write(loc, &val, size);
} else {
if (memcmp(loc, &val, size)) {
pr_warn("x86/modules: Invalid relocation target, existing value does not match expected value for type %d, loc %p, val %Lx\n",
(int)ELF64_R_TYPE(rel[i].r_info), loc, val);
pr_warn("x86/modules: Invalid relocation target, existing value does not match expected value for sec %u, idx %u, type %d, loc %lx, val %llx\n",
relsec, i, (int)ELF64_R_TYPE(rel[i].r_info),
(unsigned long)loc, val);
return -ENOEXEC;
}
write(loc, &zero, size);
@@ -179,8 +182,8 @@ static int __write_relocate_add(Elf64_Shdr *sechdrs,
return 0;
overflow:
pr_err("overflow in relocation type %d val %Lx\n",
(int)ELF64_R_TYPE(rel[i].r_info), val);
pr_err("overflow in relocation type %d val %llx sec %u idx %d\n",
(int)ELF64_R_TYPE(rel[i].r_info), val, relsec, i);
pr_err("`%s' likely not compiled with -mcmodel=kernel\n",
me->name);
return -ENOEXEC;

8
arch/x86/kernel/smpboot.c
View File

@@ -1328,11 +1328,7 @@ void __noreturn hlt_play_dead(void)
native_halt();
}
/*
* native_play_dead() is essentially a __noreturn function, but it can't
* be marked as such as the compiler may complain about it.
*/
void native_play_dead(void)
void __noreturn native_play_dead(void)
{
if (cpu_feature_enabled(X86_FEATURE_KERNEL_IBRS))
__update_spec_ctrl(0);
@@ -1351,7 +1347,7 @@ int native_cpu_disable(void)
return -ENOSYS;
}
void native_play_dead(void)
void __noreturn native_play_dead(void)
{
BUG();
}

4
drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.h
View File

@@ -53,6 +53,10 @@ extern void
usnic_uiom_interval_tree_remove(struct usnic_uiom_interval_node *node,
struct rb_root_cached *root);
extern struct usnic_uiom_interval_node *
usnic_uiom_interval_tree_subtree_search(struct usnic_uiom_interval_node *node,
unsigned long start,
unsigned long last);
extern struct usnic_uiom_interval_node *
usnic_uiom_interval_tree_iter_first(struct rb_root_cached *root,
unsigned long start,
unsigned long last);

4
drivers/staging/media/atomisp/i2c/atomisp-gc2235.c
View File

@@ -491,7 +491,7 @@ static int gc2235_s_power(struct v4l2_subdev *sd, int on)
return ret;
}
static int startup(struct v4l2_subdev *sd)
static int gc2235_startup(struct v4l2_subdev *sd)
{
struct gc2235_device *dev = to_gc2235_sensor(sd);
struct i2c_client *client = v4l2_get_subdevdata(sd);
@@ -556,7 +556,7 @@ static int gc2235_set_fmt(struct v4l2_subdev *sd,
return 0;
}
ret = startup(sd);
ret = gc2235_startup(sd);
if (ret) {
dev_err(&client->dev, "gc2235 startup err\n");
goto err;

6
drivers/staging/media/atomisp/i2c/atomisp-ov2722.c
View File

@@ -600,7 +600,7 @@ static int ov2722_s_power(struct v4l2_subdev *sd, int on)
}
/* TODO: remove it. */
static int startup(struct v4l2_subdev *sd)
static int ov2722_startup(struct v4l2_subdev *sd)
{
struct ov2722_device *dev = to_ov2722_sensor(sd);
struct i2c_client *client = v4l2_get_subdevdata(sd);
@@ -662,7 +662,7 @@ static int ov2722_set_fmt(struct v4l2_subdev *sd,
dev->pixels_per_line = dev->res->pixels_per_line;
dev->lines_per_frame = dev->res->lines_per_frame;
ret = startup(sd);
ret = ov2722_startup(sd);
if (ret) {
int i = 0;
@@ -677,7 +677,7 @@ static int ov2722_set_fmt(struct v4l2_subdev *sd,
dev_err(&client->dev, "power up failed, continue\n");
continue;
}
ret = startup(sd);
ret = ov2722_startup(sd);
if (ret) {
dev_err(&client->dev, " startup FAILED!\n");
} else {

14
drivers/tty/amiserial.c
View File

@@ -438,7 +438,7 @@ static irqreturn_t ser_tx_int(int irq, void *dev_id)
* ---------------------------------------------------------------
*/
static int startup(struct tty_struct *tty, struct serial_state *info)
static int rs_startup(struct tty_struct *tty, struct serial_state *info)
{
struct tty_port *port = &info->tport;
unsigned long flags;
@@ -513,7 +513,7 @@ errout:
* This routine will shutdown a serial port; interrupts are disabled, and
* DTR is dropped if the hangup on close termio flag is on.
*/
static void shutdown(struct tty_struct *tty, struct serial_state *info)
static void rs_shutdown(struct tty_struct *tty, struct serial_state *info)
{
unsigned long flags;
@@ -975,7 +975,7 @@ check_and_exit:
change_speed(tty, state, NULL);
}
} else
retval = startup(tty, state);
retval = rs_startup(tty, state);
tty_unlock(tty);
return retval;
}
@@ -1251,9 +1251,9 @@ static void rs_close(struct tty_struct *tty, struct file * filp)
*/
rs_wait_until_sent(tty, state->timeout);
}
shutdown(tty, state);
rs_shutdown(tty, state);
rs_flush_buffer(tty);
tty_ldisc_flush(tty);
port->tty = NULL;
@@ -1325,7 +1325,7 @@ static void rs_hangup(struct tty_struct *tty)
struct serial_state *info = tty->driver_data;
rs_flush_buffer(tty);
shutdown(tty, info);
rs_shutdown(tty, info);
info->tport.count = 0;
tty_port_set_active(&info->tport, false);
info->tport.tty = NULL;
@@ -1349,7 +1349,7 @@ static int rs_open(struct tty_struct *tty, struct file * filp)
port->tty = tty;
tty->driver_data = info;
retval = startup(tty, info);
retval = rs_startup(tty, info);
if (retval) {
return retval;
}

8
drivers/tty/serial/icom.c
View File

@@ -760,7 +760,7 @@ static void load_code(struct icom_port *icom_port)
dma_free_coherent(&dev->dev, 4096, new_page, temp_pci);
}
static int startup(struct icom_port *icom_port)
static int icom_startup(struct icom_port *icom_port)
{
unsigned long temp;
unsigned char cable_id, raw_cable_id;
@@ -832,7 +832,7 @@ unlock:
return 0;
}
static void shutdown(struct icom_port *icom_port)
static void icom_shutdown(struct icom_port *icom_port)
{
unsigned long temp;
unsigned char cmdReg;
@@ -1311,7 +1311,7 @@ static int icom_open(struct uart_port *port)
int retval;
kref_get(&icom_port->adapter->kref);
retval = startup(icom_port);
retval = icom_startup(icom_port);
if (retval) {
kref_put(&icom_port->adapter->kref, icom_kref_release);
@@ -1333,7 +1333,7 @@ static void icom_close(struct uart_port *port)
cmdReg = readb(&icom_port->dram->CmdReg);
writeb(cmdReg & ~CMD_RCV_ENABLE, &icom_port->dram->CmdReg);
shutdown(icom_port);
icom_shutdown(icom_port);
kref_put(&icom_port->adapter->kref, icom_kref_release);
}

20
drivers/tty/synclink_gt.c
View File

@@ -407,9 +407,9 @@ static void wr_reg32(struct slgt_info *info, unsigned int addr, __u32 value);
static void msc_set_vcr(struct slgt_info *info);
static int startup(struct slgt_info *info);
static int startup_hw(struct slgt_info *info);
static int block_til_ready(struct tty_struct *tty, struct file * filp,struct slgt_info *info);
static void shutdown(struct slgt_info *info);
static void shutdown_hw(struct slgt_info *info);
static void program_hw(struct slgt_info *info);
static void change_params(struct slgt_info *info);
@@ -622,7 +622,7 @@ static int open(struct tty_struct *tty, struct file *filp)
if (info->port.count == 1) {
/* 1st open on this device, init hardware */
retval = startup(info);
retval = startup_hw(info);
if (retval < 0) {
mutex_unlock(&info->port.mutex);
goto cleanup;
@@ -666,7 +666,7 @@ static void close(struct tty_struct *tty, struct file *filp)
flush_buffer(tty);
tty_ldisc_flush(tty);
shutdown(info);
shutdown_hw(info);
mutex_unlock(&info->port.mutex);
tty_port_close_end(&info->port, tty);
@@ -687,7 +687,7 @@ static void hangup(struct tty_struct *tty)
flush_buffer(tty);
mutex_lock(&info->port.mutex);
shutdown(info);
shutdown_hw(info);
spin_lock_irqsave(&info->port.lock, flags);
info->port.count = 0;
@@ -1445,7 +1445,7 @@ static int hdlcdev_open(struct net_device *dev)
spin_unlock_irqrestore(&info->netlock, flags);
/* claim resources and init adapter */
if ((rc = startup(info)) != 0) {
if ((rc = startup_hw(info)) != 0) {
spin_lock_irqsave(&info->netlock, flags);
info->netcount=0;
spin_unlock_irqrestore(&info->netlock, flags);
@@ -1455,7 +1455,7 @@ static int hdlcdev_open(struct net_device *dev)
/* generic HDLC layer open processing */
rc = hdlc_open(dev);
if (rc) {
shutdown(info);
shutdown_hw(info);
spin_lock_irqsave(&info->netlock, flags);
info->netcount = 0;
spin_unlock_irqrestore(&info->netlock, flags);
@@ -1499,7 +1499,7 @@ static int hdlcdev_close(struct net_device *dev)
netif_stop_queue(dev);
/* shutdown adapter and release resources */
shutdown(info);
shutdown_hw(info);
hdlc_close(dev);
@@ -2328,7 +2328,7 @@ static irqreturn_t slgt_interrupt(int dummy, void *dev_id)
return IRQ_HANDLED;
}
static int startup(struct slgt_info *info)
static int startup_hw(struct slgt_info *info)
{
DBGINFO(("%s startup\n", info->device_name));
@@ -2361,7 +2361,7 @@ static int startup(struct slgt_info *info)
/*
* called by close() and hangup() to shutdown hardware
*/
static void shutdown(struct slgt_info *info)
static void shutdown_hw(struct slgt_info *info)
{
unsigned long flags;

4
drivers/xen/xenbus/xenbus_xs.c
View File

@@ -410,7 +410,7 @@ static char *join(const char *dir, const char *name)
return (!buffer) ? ERR_PTR(-ENOMEM) : buffer;
}
static char **split(char *strings, unsigned int len, unsigned int *num)
static char **split_strings(char *strings, unsigned int len, unsigned int *num)
{
char *p, **ret;
@@ -448,7 +448,7 @@ char **xenbus_directory(struct xenbus_transaction t,
if (IS_ERR(strings))
return ERR_CAST(strings);
return split(strings, len, num);
return split_strings(strings, len, num);
}
EXPORT_SYMBOL_GPL(xenbus_directory);

76
include/asm-generic/vmlinux.lds.h
View File

@@ -87,39 +87,56 @@
#define ALIGN_FUNCTION() . = ALIGN(CONFIG_FUNCTION_ALIGNMENT)
/*
* LD_DEAD_CODE_DATA_ELIMINATION option enables -fdata-sections, which
* generates .data.identifier sections, which need to be pulled in with
* .data. We don't want to pull in .data..other sections, which Linux
* has defined. Same for text and bss.
* Support -ffunction-sections by matching .text and .text.*,
* but exclude '.text..*', .text.startup[.*], and .text.exit[.*].
*
* With LTO_CLANG, the linker also splits sections by default, so we need
* these macros to combine the sections during the final link.
* .text.startup and .text.startup.* are matched later by INIT_TEXT, and
* .text.exit and .text.exit.* are matched later by EXIT_TEXT, so they must be
* explicitly excluded here.
*
* With AUTOFDO_CLANG and PROPELLER_CLANG, by default, the linker splits
* text sections and regroups functions into subsections.
* Other .text.* sections that are typically grouped separately, such as
* .text.unlikely or .text.hot, must be matched explicitly before using
* TEXT_MAIN.
*
* RODATA_MAIN is not used because existing code already defines .rodata.x
* sections to be brought in with rodata.
* NOTE: builds *with* and *without* -ffunction-sections are both supported by
* this single macro. Even with -ffunction-sections, there may be some objects
* NOT compiled with the flag due to the use of a specific Makefile override
* like cflags-y or AUTOFDO_PROFILE_foo.o. So this single catchall rule is
* needed to support mixed object builds.
*
* One implication is that functions named startup(), exit(), split(),
* unlikely(), hot(), and unknown() are not allowed in the kernel due to the
* ambiguity of their section names with -ffunction-sections. For example,
* .text.startup could be __attribute__((constructor)) code in a *non*
* ffunction-sections object, which should be placed in .init.text; or it could
* be an actual function named startup() in an ffunction-sections object, which
* should be placed in .text. The build will detect and complain about any such
* ambiguously named functions.
*/
#define TEXT_MAIN \
.text \
.text.[_0-9A-Za-df-rt-z]* \
.text.s[_0-9A-Za-su-z]* .text.s .text.s.* \
.text.st[_0-9A-Zb-z]* .text.st .text.st.* \
.text.sta[_0-9A-Za-qs-z]* .text.sta .text.sta.* \
.text.star[_0-9A-Za-su-z]* .text.star .text.star.* \
.text.start[_0-9A-Za-tv-z]* .text.start .text.start.* \
.text.startu[_0-9A-Za-oq-z]* .text.startu .text.startu.* \
.text.startup[_0-9A-Za-z]* \
.text.e[_0-9A-Za-wy-z]* .text.e .text.e.* \
.text.ex[_0-9A-Za-hj-z]* .text.ex .text.ex.* \
.text.exi[_0-9A-Za-su-z]* .text.exi .text.exi.* \
.text.exit[_0-9A-Za-z]*
/*
* Support -fdata-sections by matching .data, .data.*, and others,
* but exclude '.data..*'.
*/
#if defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) || defined(CONFIG_LTO_CLANG) || \
defined(CONFIG_AUTOFDO_CLANG) || defined(CONFIG_PROPELLER_CLANG)
#define TEXT_MAIN .text .text.[0-9a-zA-Z_]*
#else
#define TEXT_MAIN .text
#endif
#if defined(CONFIG_LD_DEAD_CODE_DATA_ELIMINATION) || defined(CONFIG_LTO_CLANG)
#define DATA_MAIN .data .data.[0-9a-zA-Z_]* .data.rel.* .data..L* .data..compoundliteral* .data.$__unnamed_* .data.$L*
#define SDATA_MAIN .sdata .sdata.[0-9a-zA-Z_]*
#define RODATA_MAIN .rodata .rodata.[0-9a-zA-Z_]* .rodata..L*
#define BSS_MAIN .bss .bss.[0-9a-zA-Z_]* .bss..L* .bss..compoundliteral*
#define SBSS_MAIN .sbss .sbss.[0-9a-zA-Z_]*
#else
#define DATA_MAIN .data .data.rel .data.rel.local
#define SDATA_MAIN .sdata
#define RODATA_MAIN .rodata
#define BSS_MAIN .bss
#define SBSS_MAIN .sbss
#endif
/*
* GCC 4.5 and later have a 32 bytes section alignment for structures.
@@ -581,9 +598,8 @@ defined(CONFIG_AUTOFDO_CLANG) || defined(CONFIG_PROPELLER_CLANG)
* during second ld run in second ld pass when generating System.map
*
* TEXT_MAIN here will match symbols with a fixed pattern (for example,
* .text.hot or .text.unlikely) if dead code elimination or
* function-section is enabled. Match these symbols first before
* TEXT_MAIN to ensure they are grouped together.
* .text.hot or .text.unlikely). Match those before TEXT_MAIN to ensure
* they get grouped together.
*
* Also placing .text.hot section at the beginning of a page, this
* would help the TLB performance.
@@ -729,16 +745,16 @@ defined(CONFIG_AUTOFDO_CLANG) || defined(CONFIG_PROPELLER_CLANG)
#define INIT_TEXT \
*(.init.text .init.text.*) \
*(.text.startup)
*(.text.startup .text.startup.*)
#define EXIT_DATA \
*(.exit.data .exit.data.*) \
*(.fini_array .fini_array.*) \
*(.dtors .dtors.*) \
*(.dtors .dtors.*)
#define EXIT_TEXT \
*(.exit.text) \
*(.text.exit) \
*(.text.exit .text.exit.*)
#define EXIT_CALL \
*(.exitcall.exit)

134
include/linux/annotate.h Normal file
View File

@@ -0,0 +1,134 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _LINUX_ANNOTATE_H
#define _LINUX_ANNOTATE_H
#include <linux/objtool_types.h>
#ifdef CONFIG_OBJTOOL
#ifndef __ASSEMBLY__
#define __ASM_ANNOTATE(section, label, type) \
".pushsection " section ",\"M\", @progbits, 8\n\t" \
".long " __stringify(label) " - .\n\t" \
".long " __stringify(type) "\n\t" \
".popsection\n\t"
#define ASM_ANNOTATE_LABEL(label, type) \
__ASM_ANNOTATE(".discard.annotate_insn", label, type)
#define ASM_ANNOTATE(type) \
"911:\n\t" \
ASM_ANNOTATE_LABEL(911b, type)
#define ASM_ANNOTATE_DATA(type) \
"912:\n\t" \
__ASM_ANNOTATE(".discard.annotate_data", 912b, type)
#else /* __ASSEMBLY__ */
.macro __ANNOTATE section, type
.Lhere_\@:
.pushsection \section, "M", @progbits, 8
.long .Lhere_\@ - .
.long \type
.popsection
.endm
.macro ANNOTATE type
__ANNOTATE ".discard.annotate_insn", \type
.endm
.macro ANNOTATE_DATA type
__ANNOTATE ".discard.annotate_data", \type
.endm
#endif /* __ASSEMBLY__ */
#else /* !CONFIG_OBJTOOL */
#ifndef __ASSEMBLY__
#define ASM_ANNOTATE_LABEL(label, type) ""
#define ASM_ANNOTATE(type)
#define ASM_ANNOTATE_DATA(type)
#else /* __ASSEMBLY__ */
.macro ANNOTATE type
.endm
.macro ANNOTATE_DATA type
.endm
#endif /* __ASSEMBLY__ */
#endif /* !CONFIG_OBJTOOL */
#ifndef __ASSEMBLY__
/*
* Annotate away the various 'relocation to !ENDBR` complaints; knowing that
* these relocations will never be used for indirect calls.
*/
#define ANNOTATE_NOENDBR ASM_ANNOTATE(ANNOTYPE_NOENDBR)
#define ANNOTATE_NOENDBR_SYM(sym) asm(ASM_ANNOTATE_LABEL(sym, ANNOTYPE_NOENDBR))
/*
* This should be used immediately before an indirect jump/call. It tells
* objtool the subsequent indirect jump/call is vouched safe for retpoline
* builds.
*/
#define ANNOTATE_RETPOLINE_SAFE ASM_ANNOTATE(ANNOTYPE_RETPOLINE_SAFE)
/*
* See linux/instrumentation.h
*/
#define ANNOTATE_INSTR_BEGIN(label) ASM_ANNOTATE_LABEL(label, ANNOTYPE_INSTR_BEGIN)
#define ANNOTATE_INSTR_END(label) ASM_ANNOTATE_LABEL(label, ANNOTYPE_INSTR_END)
/*
* objtool annotation to ignore the alternatives and only consider the original
* instruction(s).
*/
#define ANNOTATE_IGNORE_ALTERNATIVE ASM_ANNOTATE(ANNOTYPE_IGNORE_ALTS)
/*
* This macro indicates that the following intra-function call is valid.
* Any non-annotated intra-function call will cause objtool to issue a warning.
*/
#define ANNOTATE_INTRA_FUNCTION_CALL ASM_ANNOTATE(ANNOTYPE_INTRA_FUNCTION_CALL)
/*
* Use objtool to validate the entry requirement that all code paths do
* VALIDATE_UNRET_END before RET.
*
* NOTE: The macro must be used at the beginning of a global symbol, otherwise
* it will be ignored.
*/
#define ANNOTATE_UNRET_BEGIN ASM_ANNOTATE(ANNOTYPE_UNRET_BEGIN)
/*
* This should be used to refer to an instruction that is considered
* terminating, like a noreturn CALL or UD2 when we know they are not -- eg
* WARN using UD2.
*/
#define ANNOTATE_REACHABLE(label) ASM_ANNOTATE_LABEL(label, ANNOTYPE_REACHABLE)
/*
* This should not be used; it annotates away CFI violations. There are a few
* valid use cases like kexec handover to the next kernel image, and there is
* no security concern there.
*
* There are also a few real issues annotated away, like EFI because we can't
* control the EFI code.
*/
#define ANNOTATE_NOCFI_SYM(sym) asm(ASM_ANNOTATE_LABEL(sym, ANNOTYPE_NOCFI))
/*
* Annotate a special section entry. This emables livepatch module generation
* to find and extract individual special section entries as needed.
*/
#define ANNOTATE_DATA_SPECIAL ASM_ANNOTATE_DATA(ANNOTYPE_DATA_SPECIAL)
#else /* __ASSEMBLY__ */
#define ANNOTATE_NOENDBR ANNOTATE type=ANNOTYPE_NOENDBR
#define ANNOTATE_RETPOLINE_SAFE ANNOTATE type=ANNOTYPE_RETPOLINE_SAFE
/* ANNOTATE_INSTR_BEGIN ANNOTATE type=ANNOTYPE_INSTR_BEGIN */
/* ANNOTATE_INSTR_END ANNOTATE type=ANNOTYPE_INSTR_END */
#define ANNOTATE_IGNORE_ALTERNATIVE ANNOTATE type=ANNOTYPE_IGNORE_ALTS
#define ANNOTATE_INTRA_FUNCTION_CALL ANNOTATE type=ANNOTYPE_INTRA_FUNCTION_CALL
#define ANNOTATE_UNRET_BEGIN ANNOTATE type=ANNOTYPE_UNRET_BEGIN
#define ANNOTATE_REACHABLE ANNOTATE type=ANNOTYPE_REACHABLE
#define ANNOTATE_NOCFI_SYM ANNOTATE type=ANNOTYPE_NOCFI
#define ANNOTATE_DATA_SPECIAL ANNOTATE_DATA type=ANNOTYPE_DATA_SPECIAL
#endif /* __ASSEMBLY__ */
#endif /* _LINUX_ANNOTATE_H */

8
include/linux/compiler.h
View File

@@ -163,7 +163,11 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
__asm__ ("" : "=r" (var) : "0" (var))
#endif
#define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __COUNTER__)
/* Format: __UNIQUE_ID_<name>_<__COUNTER__> */
#define __UNIQUE_ID(name) \
__PASTE(__UNIQUE_ID_, \
__PASTE(name, \
__PASTE(_, __COUNTER__)))
/**
* data_race - mark an expression as containing intentional data races
@@ -283,7 +287,7 @@ static inline void *offset_to_ptr(const int *off)
*/
#define ___ADDRESSABLE(sym, __attrs) \
static void * __used __attrs \
__UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)(uintptr_t)&sym;
__UNIQUE_ID(__PASTE(addressable_, sym)) = (void *)(uintptr_t)&sym;
#define __ADDRESSABLE(sym) \
___ADDRESSABLE(sym, __section(".discard.addressable"))

13
include/linux/elfnote.h
View File

@@ -60,23 +60,21 @@
#else /* !__ASSEMBLER__ */
#include <uapi/linux/elf.h>
#include <linux/compiler.h>
/*
* Use an anonymous structure which matches the shape of
* Elf{32,64}_Nhdr, but includes the name and desc data. The size and
* type of name and desc depend on the macro arguments. "name" must
* be a literal string, and "desc" must be passed by value. You may
* only define one note per line, since __LINE__ is used to generate
* unique symbols.
* be a literal string, and "desc" must be passed by value.
*/
#define _ELFNOTE_PASTE(a,b) a##b
#define _ELFNOTE(size, name, unique, type, desc) \
#define ELFNOTE(size, name, type, desc) \
static const struct { \
struct elf##size##_note _nhdr; \
unsigned char _name[sizeof(name)] \
__attribute__((aligned(sizeof(Elf##size##_Word)))); \
typeof(desc) _desc \
__attribute__((aligned(sizeof(Elf##size##_Word)))); \
} _ELFNOTE_PASTE(_note_, unique) \
} __UNIQUE_ID(note) \
__used \
__attribute__((section(".note." name), \
aligned(sizeof(Elf##size##_Word)), \
@@ -89,11 +87,10 @@
name, \
desc \
}
#define ELFNOTE(size, name, type, desc) \
_ELFNOTE(size, name, __LINE__, type, desc)
#define ELFNOTE32(name, type, desc) ELFNOTE(32, name, type, desc)
#define ELFNOTE64(name, type, desc) ELFNOTE(64, name, type, desc)
#endif /* __ASSEMBLER__ */
#endif /* _LINUX_ELFNOTE_H */

3
include/linux/init.h
View File

@@ -200,12 +200,13 @@ extern struct module __this_module;
/* Format: <modname>__<counter>_<line>_<fn> */
#define __initcall_id(fn) \
__PASTE(kmod_, \
__PASTE(__KBUILD_MODNAME, \
__PASTE(__, \
__PASTE(__COUNTER__, \
__PASTE(_, \
__PASTE(__LINE__, \
__PASTE(_, fn))))))
__PASTE(_, fn)))))))
/* Format: __<prefix>__<iid><id> */
#define __initcall_name(prefix, __iid, id) \

4
include/linux/interval_tree.h
View File

@@ -19,6 +19,10 @@ extern void
interval_tree_remove(struct interval_tree_node *node,
struct rb_root_cached *root);
extern struct interval_tree_node *
interval_tree_subtree_search(struct interval_tree_node *node,
unsigned long start, unsigned long last);
extern struct interval_tree_node *
interval_tree_iter_first(struct rb_root_cached *root,
unsigned long start, unsigned long last);

2
include/linux/interval_tree_generic.h
View File

@@ -77,7 +77,7 @@ ITSTATIC void ITPREFIX ## _remove(ITSTRUCT *node, \
* Cond2: start <= ITLAST(node) \
*/ \
\
static ITSTRUCT * \
ITSTATIC ITSTRUCT * \
ITPREFIX ## _subtree_search(ITSTRUCT *node, ITTYPE start, ITTYPE last) \
{ \
while (true) { \

25
include/linux/livepatch.h
View File

@@ -13,6 +13,7 @@
#include <linux/ftrace.h>
#include <linux/completion.h>
#include <linux/list.h>
#include <linux/livepatch_external.h>
#include <linux/livepatch_sched.h>
#if IS_ENABLED(CONFIG_LIVEPATCH)
@@ -77,30 +78,6 @@ struct klp_func {
bool transition;
};
struct klp_object;
/**
* struct klp_callbacks - pre/post live-(un)patch callback structure
* @pre_patch: executed before code patching
* @post_patch: executed after code patching
* @pre_unpatch: executed before code unpatching
* @post_unpatch: executed after code unpatching
* @post_unpatch_enabled: flag indicating if post-unpatch callback
* should run
*
* All callbacks are optional. Only the pre-patch callback, if provided,
* will be unconditionally executed. If the parent klp_object fails to
* patch for any reason, including a non-zero error status returned from
* the pre-patch callback, no further callbacks will be executed.
*/
struct klp_callbacks {
int (*pre_patch)(struct klp_object *obj);
void (*post_patch)(struct klp_object *obj);
void (*pre_unpatch)(struct klp_object *obj);
void (*post_unpatch)(struct klp_object *obj);
bool post_unpatch_enabled;
};
/**
* struct klp_object - kernel object structure for live patching
* @name: module name (or NULL for vmlinux)

76
include/linux/livepatch_external.h Normal file
View File

@@ -0,0 +1,76 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* External livepatch interfaces for patch creation tooling
*/
#ifndef _LINUX_LIVEPATCH_EXTERNAL_H_
#define _LINUX_LIVEPATCH_EXTERNAL_H_
#include <linux/types.h>
#define KLP_RELOC_SEC_PREFIX ".klp.rela."
#define KLP_SYM_PREFIX ".klp.sym."
#define __KLP_PRE_PATCH_PREFIX __klp_pre_patch_callback_
#define __KLP_POST_PATCH_PREFIX __klp_post_patch_callback_
#define __KLP_PRE_UNPATCH_PREFIX __klp_pre_unpatch_callback_
#define __KLP_POST_UNPATCH_PREFIX __klp_post_unpatch_callback_
#define KLP_PRE_PATCH_PREFIX __stringify(__KLP_PRE_PATCH_PREFIX)
#define KLP_POST_PATCH_PREFIX __stringify(__KLP_POST_PATCH_PREFIX)
#define KLP_PRE_UNPATCH_PREFIX __stringify(__KLP_PRE_UNPATCH_PREFIX)
#define KLP_POST_UNPATCH_PREFIX __stringify(__KLP_POST_UNPATCH_PREFIX)
struct klp_object;
typedef int (*klp_pre_patch_t)(struct klp_object *obj);
typedef void (*klp_post_patch_t)(struct klp_object *obj);
typedef void (*klp_pre_unpatch_t)(struct klp_object *obj);
typedef void (*klp_post_unpatch_t)(struct klp_object *obj);
/**
* struct klp_callbacks - pre/post live-(un)patch callback structure
* @pre_patch: executed before code patching
* @post_patch: executed after code patching
* @pre_unpatch: executed before code unpatching
* @post_unpatch: executed after code unpatching
* @post_unpatch_enabled: flag indicating if post-unpatch callback
* should run
*
* All callbacks are optional. Only the pre-patch callback, if provided,
* will be unconditionally executed. If the parent klp_object fails to
* patch for any reason, including a non-zero error status returned from
* the pre-patch callback, no further callbacks will be executed.
*/
struct klp_callbacks {
klp_pre_patch_t pre_patch;
klp_post_patch_t post_patch;
klp_pre_unpatch_t pre_unpatch;
klp_post_unpatch_t post_unpatch;
bool post_unpatch_enabled;
};
/*
* 'struct klp_{func,object}_ext' are compact "external" representations of
* 'struct klp_{func,object}'. They are used by objtool for livepatch
* generation. The structs are then read by the livepatch module and converted
* to the real structs before calling klp_enable_patch().
*
* TODO make these the official API for klp_enable_patch(). That should
* simplify livepatch's interface as well as its data structure lifetime
* management.
*/
struct klp_func_ext {
const char *old_name;
void *new_func;
unsigned long sympos;
};
struct klp_object_ext {
const char *name;
struct klp_func_ext *funcs;
struct klp_callbacks callbacks;
unsigned int nr_funcs;
};
#endif /* _LINUX_LIVEPATCH_EXTERNAL_H_ */

77
include/linux/livepatch_helpers.h Normal file
View File

@@ -0,0 +1,77 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _LINUX_LIVEPATCH_HELPERS_H
#define _LINUX_LIVEPATCH_HELPERS_H
/*
* Interfaces for use by livepatch patches
*/
#include <linux/syscalls.h>
#include <linux/livepatch.h>
#ifdef MODULE
#define KLP_OBJNAME __KBUILD_MODNAME
#else
#define KLP_OBJNAME vmlinux
#endif
/* Livepatch callback registration */
#define KLP_CALLBACK_PTRS ".discard.klp_callback_ptrs"
#define KLP_PRE_PATCH_CALLBACK(func) \
klp_pre_patch_t __used __section(KLP_CALLBACK_PTRS) \
__PASTE(__KLP_PRE_PATCH_PREFIX, KLP_OBJNAME) = func
#define KLP_POST_PATCH_CALLBACK(func) \
klp_post_patch_t __used __section(KLP_CALLBACK_PTRS) \
__PASTE(__KLP_POST_PATCH_PREFIX, KLP_OBJNAME) = func
#define KLP_PRE_UNPATCH_CALLBACK(func) \
klp_pre_unpatch_t __used __section(KLP_CALLBACK_PTRS) \
__PASTE(__KLP_PRE_UNPATCH_PREFIX, KLP_OBJNAME) = func
#define KLP_POST_UNPATCH_CALLBACK(func) \
klp_post_unpatch_t __used __section(KLP_CALLBACK_PTRS) \
__PASTE(__KLP_POST_UNPATCH_PREFIX, KLP_OBJNAME) = func
/*
* Replace static_call() usage with this macro when create-diff-object
* recommends it due to the original static call key living in a module.
*
* This converts the static call to a regular indirect call.
*/
#define KLP_STATIC_CALL(name) \
((typeof(STATIC_CALL_TRAMP(name))*)(STATIC_CALL_KEY(name).func))
/* Syscall patching */
#define KLP_SYSCALL_DEFINE1(name, ...) KLP_SYSCALL_DEFINEx(1, _##name, __VA_ARGS__)
#define KLP_SYSCALL_DEFINE2(name, ...) KLP_SYSCALL_DEFINEx(2, _##name, __VA_ARGS__)
#define KLP_SYSCALL_DEFINE3(name, ...) KLP_SYSCALL_DEFINEx(3, _##name, __VA_ARGS__)
#define KLP_SYSCALL_DEFINE4(name, ...) KLP_SYSCALL_DEFINEx(4, _##name, __VA_ARGS__)
#define KLP_SYSCALL_DEFINE5(name, ...) KLP_SYSCALL_DEFINEx(5, _##name, __VA_ARGS__)
#define KLP_SYSCALL_DEFINE6(name, ...) KLP_SYSCALL_DEFINEx(6, _##name, __VA_ARGS__)
#define KLP_SYSCALL_DEFINEx(x, sname, ...) \
__KLP_SYSCALL_DEFINEx(x, sname, __VA_ARGS__)
#ifdef CONFIG_X86_64
// TODO move this to arch/x86/include/asm/syscall_wrapper.h and share code
#define __KLP_SYSCALL_DEFINEx(x, name, ...) \
static long __se_sys##name(__MAP(x,__SC_LONG,__VA_ARGS__)); \
static inline long __klp_do_sys##name(__MAP(x,__SC_DECL,__VA_ARGS__));\
__X64_SYS_STUBx(x, name, __VA_ARGS__) \
__IA32_SYS_STUBx(x, name, __VA_ARGS__) \
static long __se_sys##name(__MAP(x,__SC_LONG,__VA_ARGS__)) \
{ \
long ret = __klp_do_sys##name(__MAP(x,__SC_CAST,__VA_ARGS__));\
__MAP(x,__SC_TEST,__VA_ARGS__); \
__PROTECT(x, ret,__MAP(x,__SC_ARGS,__VA_ARGS__)); \
return ret; \
} \
static inline long __klp_do_sys##name(__MAP(x,__SC_DECL,__VA_ARGS__))
#endif
#endif /* _LINUX_LIVEPATCH_HELPERS_H */

2
include/linux/mm.h
View File

@@ -3376,6 +3376,8 @@ void vma_interval_tree_insert_after(struct vm_area_struct *node,
struct rb_root_cached *root);
void vma_interval_tree_remove(struct vm_area_struct *node,
struct rb_root_cached *root);
struct vm_area_struct *vma_interval_tree_subtree_search(struct vm_area_struct *node,
unsigned long start, unsigned long last);
struct vm_area_struct *vma_interval_tree_iter_first(struct rb_root_cached *root,
unsigned long start, unsigned long last);
struct vm_area_struct *vma_interval_tree_iter_next(struct vm_area_struct *node,

3
include/linux/module.h
View File

@@ -251,10 +251,11 @@ struct module_kobject *lookup_or_create_module_kobject(const char *name);
*/
#define __mod_device_table(type, name) \
__PASTE(__mod_device_table__, \
__PASTE(kmod_, \
__PASTE(__KBUILD_MODNAME, \
__PASTE(__, \
__PASTE(type, \
__PASTE(__, name)))))
__PASTE(__, name))))))
/* Creates an alias so file2alias.c can find device table. */
#define MODULE_DEVICE_TABLE(type, name) \

96
include/linux/objtool.h
View File

@@ -3,16 +3,16 @@
#define _LINUX_OBJTOOL_H
#include <linux/objtool_types.h>
#include <linux/annotate.h>
#ifdef CONFIG_OBJTOOL
#include <asm/asm.h>
#ifndef __ASSEMBLY__
#define UNWIND_HINT(type, sp_reg, sp_offset, signal) \
#define UNWIND_HINT(type, sp_reg, sp_offset, signal) \
"987: \n\t" \
".pushsection .discard.unwind_hints\n\t" \
ANNOTATE_DATA_SPECIAL \
/* struct unwind_hint */ \
".long 987b - .\n\t" \
".short " __stringify(sp_offset) "\n\t" \
@@ -53,16 +53,6 @@
#define __ASM_BREF(label) label ## b
#define __ASM_ANNOTATE(label, type) \
".pushsection .discard.annotate_insn,\"M\",@progbits,8\n\t" \
".long " __stringify(label) " - .\n\t" \
".long " __stringify(type) "\n\t" \
".popsection\n\t"
#define ASM_ANNOTATE(type) \
"911:\n\t" \
__ASM_ANNOTATE(911b, type)
#else /* __ASSEMBLY__ */
/*
@@ -89,6 +79,7 @@
.macro UNWIND_HINT type:req sp_reg=0 sp_offset=0 signal=0
.Lhere_\@:
.pushsection .discard.unwind_hints
ANNOTATE_DATA_SPECIAL
/* struct unwind_hint */
.long .Lhere_\@ - .
.short \sp_offset
@@ -101,7 +92,7 @@
.macro STACK_FRAME_NON_STANDARD func:req
.pushsection .discard.func_stack_frame_non_standard, "aw"
.long \func - .
.quad \func
.popsection
.endm
@@ -111,14 +102,6 @@
#endif
.endm
.macro ANNOTATE type:req
.Lhere_\@:
.pushsection .discard.annotate_insn,"M",@progbits,8
.long .Lhere_\@ - .
.long \type
.popsection
.endm
#endif /* __ASSEMBLY__ */
#else /* !CONFIG_OBJTOOL */
@@ -128,84 +111,15 @@
#define UNWIND_HINT(type, sp_reg, sp_offset, signal) "\n\t"
#define STACK_FRAME_NON_STANDARD(func)
#define STACK_FRAME_NON_STANDARD_FP(func)
#define __ASM_ANNOTATE(label, type) ""
#define ASM_ANNOTATE(type)
#else
.macro UNWIND_HINT type:req sp_reg=0 sp_offset=0 signal=0
.endm
.macro STACK_FRAME_NON_STANDARD func:req
.endm
.macro ANNOTATE type:req
.endm
#endif
#endif /* CONFIG_OBJTOOL */
#ifndef __ASSEMBLY__
/*
* Annotate away the various 'relocation to !ENDBR` complaints; knowing that
* these relocations will never be used for indirect calls.
*/
#define ANNOTATE_NOENDBR ASM_ANNOTATE(ANNOTYPE_NOENDBR)
#define ANNOTATE_NOENDBR_SYM(sym) asm(__ASM_ANNOTATE(sym, ANNOTYPE_NOENDBR))
/*
* This should be used immediately before an indirect jump/call. It tells
* objtool the subsequent indirect jump/call is vouched safe for retpoline
* builds.
*/
#define ANNOTATE_RETPOLINE_SAFE ASM_ANNOTATE(ANNOTYPE_RETPOLINE_SAFE)
/*
* See linux/instrumentation.h
*/
#define ANNOTATE_INSTR_BEGIN(label) __ASM_ANNOTATE(label, ANNOTYPE_INSTR_BEGIN)
#define ANNOTATE_INSTR_END(label) __ASM_ANNOTATE(label, ANNOTYPE_INSTR_END)
/*
* objtool annotation to ignore the alternatives and only consider the original
* instruction(s).
*/
#define ANNOTATE_IGNORE_ALTERNATIVE ASM_ANNOTATE(ANNOTYPE_IGNORE_ALTS)
/*
* This macro indicates that the following intra-function call is valid.
* Any non-annotated intra-function call will cause objtool to issue a warning.
*/
#define ANNOTATE_INTRA_FUNCTION_CALL ASM_ANNOTATE(ANNOTYPE_INTRA_FUNCTION_CALL)
/*
* Use objtool to validate the entry requirement that all code paths do
* VALIDATE_UNRET_END before RET.
*
* NOTE: The macro must be used at the beginning of a global symbol, otherwise
* it will be ignored.
*/
#define ANNOTATE_UNRET_BEGIN ASM_ANNOTATE(ANNOTYPE_UNRET_BEGIN)
/*
* This should be used to refer to an instruction that is considered
* terminating, like a noreturn CALL or UD2 when we know they are not -- eg
* WARN using UD2.
*/
#define ANNOTATE_REACHABLE(label) __ASM_ANNOTATE(label, ANNOTYPE_REACHABLE)
/*
* This should not be used; it annotates away CFI violations. There are a few
* valid use cases like kexec handover to the next kernel image, and there is
* no security concern there.
*
* There are also a few real issues annotated away, like EFI because we can't
* control the EFI code.
*/
#define ANNOTATE_NOCFI_SYM(sym) asm(__ASM_ANNOTATE(sym, ANNOTYPE_NOCFI))
#else
#define ANNOTATE_NOENDBR ANNOTATE type=ANNOTYPE_NOENDBR
#define ANNOTATE_RETPOLINE_SAFE ANNOTATE type=ANNOTYPE_RETPOLINE_SAFE
/* ANNOTATE_INSTR_BEGIN ANNOTATE type=ANNOTYPE_INSTR_BEGIN */
/* ANNOTATE_INSTR_END ANNOTATE type=ANNOTYPE_INSTR_END */
#define ANNOTATE_IGNORE_ALTERNATIVE ANNOTATE type=ANNOTYPE_IGNORE_ALTS
#define ANNOTATE_INTRA_FUNCTION_CALL ANNOTATE type=ANNOTYPE_INTRA_FUNCTION_CALL
#define ANNOTATE_UNRET_BEGIN ANNOTATE type=ANNOTYPE_UNRET_BEGIN
#define ANNOTATE_REACHABLE ANNOTATE type=ANNOTYPE_REACHABLE
#define ANNOTATE_NOCFI_SYM ANNOTATE type=ANNOTYPE_NOCFI
#endif
#if defined(CONFIG_NOINSTR_VALIDATION) && \
(defined(CONFIG_MITIGATION_UNRET_ENTRY) || defined(CONFIG_MITIGATION_SRSO))
#define VALIDATE_UNRET_BEGIN ANNOTATE_UNRET_BEGIN

2
include/linux/objtool_types.h
View File

@@ -67,4 +67,6 @@ struct unwind_hint {
#define ANNOTYPE_REACHABLE 8
#define ANNOTYPE_NOCFI 9
#define ANNOTYPE_DATA_SPECIAL 1
#endif /* _LINUX_OBJTOOL_TYPES_H */

12
kernel/livepatch/Kconfig
View File

@@ -18,3 +18,15 @@ config LIVEPATCH
module uses the interface provided by this option to register
a patch, causing calls to patched functions to be redirected
to new function code contained in the patch module.
config HAVE_KLP_BUILD
bool
help
Arch supports klp-build
config KLP_BUILD
def_bool y
depends on LIVEPATCH && HAVE_KLP_BUILD
select OBJTOOL
help
Enable klp-build support

8
kernel/livepatch/core.c
View File

@@ -217,14 +217,14 @@ static int klp_resolve_symbols(Elf_Shdr *sechdrs, const char *strtab,
for (i = 0; i < relasec->sh_size / sizeof(Elf_Rela); i++) {
sym = (Elf_Sym *)sechdrs[symndx].sh_addr + ELF_R_SYM(relas[i].r_info);
if (sym->st_shndx != SHN_LIVEPATCH) {
pr_err("symbol %s is not marked as a livepatch symbol\n",
strtab + sym->st_name);
pr_err("symbol %s at rela sec %u idx %d is not marked as a livepatch symbol\n",
strtab + sym->st_name, symndx, i);
return -EINVAL;
}
/* Format: .klp.sym.sym_objname.sym_name,sympos */
cnt = sscanf(strtab + sym->st_name,
".klp.sym.%55[^.].%511[^,],%lu",
KLP_SYM_PREFIX "%55[^.].%511[^,],%lu",
sym_objname, sym_name, &sympos);
if (cnt != 3) {
pr_err("symbol %s has an incorrectly formatted name\n",
@@ -303,7 +303,7 @@ static int klp_write_section_relocs(struct module *pmod, Elf_Shdr *sechdrs,
* See comment in klp_resolve_symbols() for an explanation
* of the selected field width value.
*/
cnt = sscanf(shstrtab + sec->sh_name, ".klp.rela.%55[^.]",
cnt = sscanf(shstrtab + sec->sh_name, KLP_RELOC_SEC_PREFIX "%55[^.]",
sec_objname);
if (cnt != 1) {
pr_err("section %s has an incorrectly formatted name\n",

1
lib/interval_tree.c
View File

@@ -13,6 +13,7 @@ INTERVAL_TREE_DEFINE(struct interval_tree_node, rb,
EXPORT_SYMBOL_GPL(interval_tree_insert);
EXPORT_SYMBOL_GPL(interval_tree_remove);
EXPORT_SYMBOL_GPL(interval_tree_subtree_search);
EXPORT_SYMBOL_GPL(interval_tree_iter_first);
EXPORT_SYMBOL_GPL(interval_tree_iter_next);

6
scripts/Makefile.lib
View File

@@ -20,7 +20,7 @@ name-fix-token = $(subst $(comma),_,$(subst -,_,$1))
name-fix = $(call stringify,$(call name-fix-token,$1))
basename_flags = -DKBUILD_BASENAME=$(call name-fix,$(basetarget))
modname_flags = -DKBUILD_MODNAME=$(call name-fix,$(modname)) \
-D__KBUILD_MODNAME=kmod_$(call name-fix-token,$(modname))
-D__KBUILD_MODNAME=$(call name-fix-token,$(modname))
modfile_flags = -DKBUILD_MODFILE=$(call stringify,$(modfile))
_c_flags = $(filter-out $(CFLAGS_REMOVE_$(target-stem).o), \
@@ -191,13 +191,13 @@ objtool-args-$(CONFIG_HAVE_STATIC_CALL_INLINE) += --static-call
objtool-args-$(CONFIG_HAVE_UACCESS_VALIDATION) += --uaccess
objtool-args-$(or $(CONFIG_GCOV_KERNEL),$(CONFIG_KCOV)) += --no-unreachable
objtool-args-$(CONFIG_PREFIX_SYMBOLS) += --prefix=$(CONFIG_FUNCTION_PADDING_BYTES)
objtool-args-$(CONFIG_OBJTOOL_WERROR) += --Werror
objtool-args-$(CONFIG_OBJTOOL_WERROR) += --werror
objtool-args = $(objtool-args-y) \
$(if $(delay-objtool), --link) \
$(if $(part-of-module), --module)
delay-objtool := $(or $(CONFIG_LTO_CLANG),$(CONFIG_X86_KERNEL_IBT))
delay-objtool := $(or $(CONFIG_LTO_CLANG),$(CONFIG_X86_KERNEL_IBT),$(CONFIG_KLP_BUILD))
cmd_objtool = $(if $(objtool-enabled), ; $(objtool) $(objtool-args) $@)
cmd_gen_objtooldep = $(if $(objtool-enabled), { echo ; echo '$@: $$(wildcard $(objtool))' ; } >> $(dot-target).cmd)

6
scripts/Makefile.vmlinux_o
View File

@@ -41,7 +41,7 @@ objtool-enabled := $(or $(delay-objtool),$(CONFIG_NOINSTR_VALIDATION))
ifeq ($(delay-objtool),y)
vmlinux-objtool-args-y += $(objtool-args-y)
else
vmlinux-objtool-args-$(CONFIG_OBJTOOL_WERROR) += --Werror
vmlinux-objtool-args-$(CONFIG_OBJTOOL_WERROR) += --werror
endif
vmlinux-objtool-args-$(CONFIG_NOINSTR_VALIDATION) += --noinstr \
@@ -63,11 +63,15 @@ quiet_cmd_ld_vmlinux.o = LD $@
--start-group $(KBUILD_VMLINUX_LIBS) --end-group \
$(cmd_objtool)
cmd_check_function_names = $(srctree)/scripts/check-function-names.sh $@
define rule_ld_vmlinux.o
$(call cmd_and_savecmd,ld_vmlinux.o)
$(call cmd,gen_objtooldep)
$(call cmd,check_function_names)
endef
vmlinux.o: $(initcalls-lds) vmlinux.a $(KBUILD_VMLINUX_LIBS) FORCE
$(call if_changed_rule,ld_vmlinux.o)

25
scripts/check-function-names.sh Executable file
View File

@@ -0,0 +1,25 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0
#
# Certain function names are disallowed due to section name ambiguities
# introduced by -ffunction-sections.
#
# See the comment above TEXT_MAIN in include/asm-generic/vmlinux.lds.h.
objfile="$1"
if [ ! -f "$objfile" ]; then
echo "usage: $0 <file.o>" >&2
exit 1
fi
bad_symbols=$(nm "$objfile" | awk '$2 ~ /^[TtWw]$/ {print $3}' | grep -E '^(startup|exit|split|unlikely|hot|unknown)(\.|$)')
if [ -n "$bad_symbols" ]; then
echo "$bad_symbols" | while read -r sym; do
echo "$objfile: error: $sym() function name creates ambiguity with -ffunction-sections" >&2
done
exit 1
fi
exit 0

19
scripts/faddr2line
View File

@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash
# SPDX-License-Identifier: GPL-2.0
#
# Translate stack dump function offsets.
@@ -76,6 +76,10 @@ ADDR2LINE="${UTIL_PREFIX}addr2line${UTIL_SUFFIX}"
AWK="awk"
GREP="grep"
# Enforce ASCII-only output from tools like readelf
# ensuring sed processes strings correctly.
export LANG=C
command -v ${AWK} >/dev/null 2>&1 || die "${AWK} isn't installed"
command -v ${READELF} >/dev/null 2>&1 || die "${READELF} isn't installed"
command -v ${ADDR2LINE} >/dev/null 2>&1 || die "${ADDR2LINE} isn't installed"
@@ -107,14 +111,19 @@ find_dir_prefix() {
run_readelf() {
local objfile=$1
local out=$(${READELF} --file-header --section-headers --symbols --wide $objfile)
local tmpfile
tmpfile=$(mktemp)
${READELF} --file-header --section-headers --symbols --wide "$objfile" > "$tmpfile"
# This assumes that readelf first prints the file header, then the section headers, then the symbols.
# Note: It seems that GNU readelf does not prefix section headers with the "There are X section headers"
# line when multiple options are given, so let's also match with the "Section Headers:" line.
ELF_FILEHEADER=$(echo "${out}" | sed -n '/There are [0-9]* section headers, starting at offset\|Section Headers:/q;p')
ELF_SECHEADERS=$(echo "${out}" | sed -n '/There are [0-9]* section headers, starting at offset\|Section Headers:/,$p' | sed -n '/Symbol table .* contains [0-9]* entries:/q;p')
ELF_SYMS=$(echo "${out}" | sed -n '/Symbol table .* contains [0-9]* entries:/,$p')
ELF_FILEHEADER=$(sed -n '/There are [0-9]* section headers, starting at offset\|Section Headers:/q;p' "$tmpfile")
ELF_SECHEADERS=$(sed -n '/There are [0-9]* section headers, starting at offset\|Section Headers:/,$p' "$tmpfile" | sed -n '/Symbol table .* contains [0-9]* entries:/q;p')
ELF_SYMS=$(sed -n '/Symbol table .* contains [0-9]* entries:/,$p' "$tmpfile")
rm -f -- "$tmpfile"
}
check_vmlinux() {

3
scripts/link-vmlinux.sh
View File

@@ -60,7 +60,8 @@ vmlinux_link()
# skip output file argument
shift
if is_enabled CONFIG_LTO_CLANG || is_enabled CONFIG_X86_KERNEL_IBT; then
if is_enabled CONFIG_LTO_CLANG || is_enabled CONFIG_X86_KERNEL_IBT ||
is_enabled CONFIG_KLP_BUILD; then
# Use vmlinux.o instead of performing the slow LTO link again.
objs=vmlinux.o
libs=

79
scripts/livepatch/fix-patch-lines Executable file
View File

@@ -0,0 +1,79 @@
#!/usr/bin/awk -f
# SPDX-License-Identifier: GPL-2.0
#
# Use #line directives to preserve original __LINE__ numbers across patches to
# avoid unwanted compilation changes.
BEGIN {
in_hunk = 0
skip = 0
}
/^--- / {
skip = $2 !~ /\.(c|h)$/
print
next
}
/^@@/ {
if (skip) {
print
next
}
in_hunk = 1
# @@ -1,3 +1,4 @@:
# 1: line number in old file
# 3: how many lines the hunk covers in old file
# 1: line number in new file
# 4: how many lines the hunk covers in new file
match($0, /^@@ -([0-9]+)(,([0-9]+))? \+([0-9]+)(,([0-9]+))? @@/, m)
# Set 'cur' to the old file's line number at the start of the hunk. It
# gets incremented for every context line and every line removal, so
# that it always represents the old file's current line number.
cur = m[1]
# last = last line number of current hunk
last = cur + (m[3] ? m[3] : 1) - 1
need_line_directive = 0
print
next
}
{
if (skip || !in_hunk || $0 ~ /^\\ No newline at end of file/) {
print
next
}
# change line
if ($0 ~ /^[+-]/) {
# inject #line after this group of changes
need_line_directive = 1
if ($0 ~ /^-/)
cur++
print
next
}
# If this is the first context line after a group of changes, inject
# the #line directive to force the compiler to correct the line
# numbering to match the original file.
if (need_line_directive) {
print "+#line " cur
need_line_directive = 0
}
if (cur == last)
in_hunk = 0
cur++
print
}

108
scripts/livepatch/init.c Normal file
View File

@@ -0,0 +1,108 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Init code for a livepatch kernel module
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/kernel.h>
#include <linux/slab.h>
#include <linux/livepatch.h>
extern struct klp_object_ext __start_klp_objects[];
extern struct klp_object_ext __stop_klp_objects[];
static struct klp_patch *patch;
static int __init livepatch_mod_init(void)
{
struct klp_object *objs;
unsigned int nr_objs;
int ret;
nr_objs = __stop_klp_objects - __start_klp_objects;
if (!nr_objs) {
pr_err("nothing to patch!\n");
ret = -EINVAL;
goto err;
}
patch = kzalloc(sizeof(*patch), GFP_KERNEL);
if (!patch) {
ret = -ENOMEM;
goto err;
}
objs = kzalloc(sizeof(struct klp_object) * (nr_objs + 1), GFP_KERNEL);
if (!objs) {
ret = -ENOMEM;
goto err_free_patch;
}
for (int i = 0; i < nr_objs; i++) {
struct klp_object_ext *obj_ext = __start_klp_objects + i;
struct klp_func_ext *funcs_ext = obj_ext->funcs;
unsigned int nr_funcs = obj_ext->nr_funcs;
struct klp_func *funcs = objs[i].funcs;
struct klp_object *obj = objs + i;
funcs = kzalloc(sizeof(struct klp_func) * (nr_funcs + 1), GFP_KERNEL);
if (!funcs) {
ret = -ENOMEM;
for (int j = 0; j < i; j++)
kfree(objs[i].funcs);
goto err_free_objs;
}
for (int j = 0; j < nr_funcs; j++) {
funcs[j].old_name = funcs_ext[j].old_name;
funcs[j].new_func = funcs_ext[j].new_func;
funcs[j].old_sympos = funcs_ext[j].sympos;
}
obj->name = obj_ext->name;
obj->funcs = funcs;
memcpy(&obj->callbacks, &obj_ext->callbacks, sizeof(struct klp_callbacks));
}
patch->mod = THIS_MODULE;
patch->objs = objs;
/* TODO patch->states */
#ifdef KLP_NO_REPLACE
patch->replace = false;
#else
patch->replace = true;
#endif
return klp_enable_patch(patch);
err_free_objs:
kfree(objs);
err_free_patch:
kfree(patch);
err:
return ret;
}
static void __exit livepatch_mod_exit(void)
{
unsigned int nr_objs;
nr_objs = __stop_klp_objects - __start_klp_objects;
for (int i = 0; i < nr_objs; i++)
kfree(patch->objs[i].funcs);
kfree(patch->objs);
kfree(patch);
}
module_init(livepatch_mod_init);
module_exit(livepatch_mod_exit);
MODULE_LICENSE("GPL");
MODULE_INFO(livepatch, "Y");
MODULE_DESCRIPTION("Livepatch module");

831
scripts/livepatch/klp-build Executable file
View File

@@ -0,0 +1,831 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0
#
# Build a livepatch module
# shellcheck disable=SC1090,SC2155
if (( BASH_VERSINFO[0] < 4 || \
(BASH_VERSINFO[0] == 4 && BASH_VERSINFO[1] < 4) )); then
echo "error: this script requires bash 4.4+" >&2
exit 1
fi
set -o errexit
set -o errtrace
set -o pipefail
set -o nounset
# Allow doing 'cmd | mapfile -t array' instead of 'mapfile -t array < <(cmd)'.
# This helps keep execution in pipes so pipefail+errexit can catch errors.
shopt -s lastpipe
unset DEBUG_CLONE DIFF_CHECKSUM SKIP_CLEANUP XTRACE
REPLACE=1
SHORT_CIRCUIT=0
JOBS="$(getconf _NPROCESSORS_ONLN)"
VERBOSE="-s"
shopt -o xtrace | grep -q 'on' && XTRACE=1
# Avoid removing the previous $TMP_DIR until args have been fully processed.
KEEP_TMP=1
SCRIPT="$(basename "$0")"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
FIX_PATCH_LINES="$SCRIPT_DIR/fix-patch-lines"
SRC="$(pwd)"
OBJ="$(pwd)"
CONFIG="$OBJ/.config"
TMP_DIR="$OBJ/klp-tmp"
ORIG_DIR="$TMP_DIR/orig"
PATCHED_DIR="$TMP_DIR/patched"
DIFF_DIR="$TMP_DIR/diff"
KMOD_DIR="$TMP_DIR/kmod"
STASH_DIR="$TMP_DIR/stash"
TIMESTAMP="$TMP_DIR/timestamp"
PATCH_TMP_DIR="$TMP_DIR/tmp"
KLP_DIFF_LOG="$DIFF_DIR/diff.log"
grep0() {
command grep "$@" || true
}
status() {
echo "$*"
}
warn() {
echo "error: $SCRIPT: $*" >&2
}
die() {
warn "$@"
exit 1
}
declare -a STASHED_FILES
stash_file() {
local file="$1"
local rel_file="${file#"$SRC"/}"
[[ ! -e "$file" ]] && die "no file to stash: $file"
mkdir -p "$STASH_DIR/$(dirname "$rel_file")"
cp -f "$file" "$STASH_DIR/$rel_file"
STASHED_FILES+=("$rel_file")
}
restore_files() {
local file
for file in "${STASHED_FILES[@]}"; do
mv -f "$STASH_DIR/$file" "$SRC/$file" || warn "can't restore file: $file"
done
STASHED_FILES=()
}
cleanup() {
set +o nounset
revert_patches "--recount"
restore_files
[[ "$KEEP_TMP" -eq 0 ]] && rm -rf "$TMP_DIR"
return 0
}
trap_err() {
warn "line ${BASH_LINENO[0]}: '$BASH_COMMAND'"
}
trap cleanup EXIT INT TERM HUP
trap trap_err ERR
__usage() {
cat <<EOF
Usage: $SCRIPT [OPTIONS] PATCH_FILE(s)
Generate a livepatch module.
Options:
-f, --show-first-changed Show address of first changed instruction
-j, --jobs=<jobs> Build jobs to run simultaneously [default: $JOBS]
-o, --output=<file.ko> Output file [default: livepatch-<patch-name>.ko]
--no-replace Disable livepatch atomic replace
-v, --verbose Pass V=1 to kernel/module builds
Advanced Options:
-d, --debug Show symbol/reloc cloning decisions
-S, --short-circuit=STEP Start at build step (requires prior --keep-tmp)
1|orig Build original kernel (default)
2|patched Build patched kernel
3|diff Diff objects
4|kmod Build patch module
-T, --keep-tmp Preserve tmp dir on exit
EOF
}
usage() {
__usage >&2
}
process_args() {
local keep_tmp=0
local short
local long
local args
short="hfj:o:vdS:T"
long="help,show-first-changed,jobs:,output:,no-replace,verbose,debug,short-circuit:,keep-tmp"
args=$(getopt --options "$short" --longoptions "$long" -- "$@") || {
echo; usage; exit
}
eval set -- "$args"
while true; do
case "$1" in
-h | --help)
usage
exit 0
;;
-f | --show-first-changed)
DIFF_CHECKSUM=1
shift
;;
-j | --jobs)
JOBS="$2"
shift 2
;;
-o | --output)
[[ "$2" != *.ko ]] && die "output filename should end with .ko"
OUTFILE="$2"
NAME="$(basename "$OUTFILE")"
NAME="${NAME%.ko}"
NAME="$(module_name_string "$NAME")"
shift 2
;;
--no-replace)
REPLACE=0
shift
;;
-v | --verbose)
VERBOSE="V=1"
shift
;;
-d | --debug)
DEBUG_CLONE=1
keep_tmp=1
shift
;;
-S | --short-circuit)
[[ ! -d "$TMP_DIR" ]] && die "--short-circuit requires preserved klp-tmp dir"
keep_tmp=1
case "$2" in
1 | orig) SHORT_CIRCUIT=1; ;;
2 | patched) SHORT_CIRCUIT=2; ;;
3 | diff) SHORT_CIRCUIT=3; ;;
4 | mod) SHORT_CIRCUIT=4; ;;
*) die "invalid short-circuit step '$2'" ;;
esac
shift 2
;;
-T | --keep-tmp)
keep_tmp=1
shift
;;
--)
shift
break
;;
*)
usage
exit 1
;;
esac
done
if [[ $# -eq 0 ]]; then
usage
exit 1
fi
KEEP_TMP="$keep_tmp"
PATCHES=("$@")
}
# temporarily disable xtrace for especially verbose code
xtrace_save() {
[[ -v XTRACE ]] && set +x
return 0
}
xtrace_restore() {
[[ -v XTRACE ]] && set -x
return 0
}
validate_config() {
xtrace_save "reading .config"
source "$CONFIG" || die "no .config file in $(dirname "$CONFIG")"
xtrace_restore
[[ -v CONFIG_LIVEPATCH ]] || \
die "CONFIG_LIVEPATCH not enabled"
[[ -v CONFIG_KLP_BUILD ]] || \
die "CONFIG_KLP_BUILD not enabled"
[[ -v CONFIG_GCC_PLUGIN_LATENT_ENTROPY ]] && \
die "kernel option 'CONFIG_GCC_PLUGIN_LATENT_ENTROPY' not supported"
[[ -v CONFIG_GCC_PLUGIN_RANDSTRUCT ]] && \
die "kernel option 'CONFIG_GCC_PLUGIN_RANDSTRUCT' not supported"
return 0
}
# Only allow alphanumerics and '_' and '-' in the module name. Everything else
# is replaced with '-'. Also truncate to 55 chars so the full name + NUL
# terminator fits in the kernel's 56-byte module name array.
module_name_string() {
echo "${1//[^a-zA-Z0-9_-]/-}" | cut -c 1-55
}
# If the module name wasn't specified on the cmdline with --output, give it a
# name based on the patch name.
set_module_name() {
[[ -v NAME ]] && return 0
if [[ "${#PATCHES[@]}" -eq 1 ]]; then
NAME="$(basename "${PATCHES[0]}")"
NAME="${NAME%.*}"
else
NAME="patch"
fi
NAME="livepatch-$NAME"
NAME="$(module_name_string "$NAME")"
OUTFILE="$NAME.ko"
}
# Hardcode the value printed by the localversion script to prevent patch
# application from appending it with '+' due to a dirty git working tree.
set_kernelversion() {
local file="$SRC/scripts/setlocalversion"
local localversion
stash_file "$file"
localversion="$(cd "$SRC" && make --no-print-directory kernelversion)"
localversion="$(cd "$SRC" && KERNELVERSION="$localversion" ./scripts/setlocalversion)"
[[ -z "$localversion" ]] && die "setlocalversion failed"
sed -i "2i echo $localversion; exit 0" scripts/setlocalversion
}
get_patch_files() {
local patch="$1"
grep0 -E '^(--- |\+\+\+ )' "$patch" \
| gawk '{print $2}' \
| sed 's|^[^/]*/||' \
| sort -u
}
# Make sure git re-stats the changed files
git_refresh() {
local patch="$1"
local files=()
[[ ! -e "$SRC/.git" ]] && return
get_patch_files "$patch" | mapfile -t files
(
cd "$SRC"
git update-index -q --refresh -- "${files[@]}"
)
}
check_unsupported_patches() {
local patch
for patch in "${PATCHES[@]}"; do
local files=()
get_patch_files "$patch" | mapfile -t files
for file in "${files[@]}"; do
case "$file" in
lib/*|*.S)
die "unsupported patch to $file"
;;
esac
done
done
}
apply_patch() {
local patch="$1"
shift
local extra_args=("$@")
[[ ! -f "$patch" ]] && die "$patch doesn't exist"
(
cd "$SRC"
# The sed strips the version signature from 'git format-patch',
# otherwise 'git apply --recount' warns.
sed -n '/^-- /q;p' "$patch" |
git apply "${extra_args[@]}"
)
APPLIED_PATCHES+=("$patch")
}
revert_patch() {
local patch="$1"
shift
local extra_args=("$@")
local tmp=()
(
cd "$SRC"
sed -n '/^-- /q;p' "$patch" |
git apply --reverse "${extra_args[@]}"
)
git_refresh "$patch"
for p in "${APPLIED_PATCHES[@]}"; do
[[ "$p" == "$patch" ]] && continue
tmp+=("$p")
done
APPLIED_PATCHES=("${tmp[@]}")
}
apply_patches() {
local patch
for patch in "${PATCHES[@]}"; do
apply_patch "$patch"
done
}
revert_patches() {
local extra_args=("$@")
local patches=("${APPLIED_PATCHES[@]}")
for (( i=${#patches[@]}-1 ; i>=0 ; i-- )) ; do
revert_patch "${patches[$i]}" "${extra_args[@]}"
done
APPLIED_PATCHES=()
}
validate_patches() {
check_unsupported_patches
apply_patches
revert_patches
}
do_init() {
# We're not yet smart enough to handle anything other than in-tree
# builds in pwd.
[[ ! "$SRC" -ef "$SCRIPT_DIR/../.." ]] && die "please run from the kernel root directory"
[[ ! "$OBJ" -ef "$SCRIPT_DIR/../.." ]] && die "please run from the kernel root directory"
(( SHORT_CIRCUIT <= 1 )) && rm -rf "$TMP_DIR"
mkdir -p "$TMP_DIR"
APPLIED_PATCHES=()
[[ -x "$FIX_PATCH_LINES" ]] || die "can't find fix-patch-lines"
validate_config
set_module_name
set_kernelversion
}
# Refresh the patch hunk headers, specifically the line numbers and counts.
refresh_patch() {
local patch="$1"
local tmpdir="$PATCH_TMP_DIR"
local files=()
rm -rf "$tmpdir"
mkdir -p "$tmpdir/a"
mkdir -p "$tmpdir/b"
# Get all source files affected by the patch
get_patch_files "$patch" | mapfile -t files
# Copy orig source files to 'a'
( cd "$SRC" && echo "${files[@]}" | xargs cp --parents --target-directory="$tmpdir/a" )
# Copy patched source files to 'b'
apply_patch "$patch" --recount
( cd "$SRC" && echo "${files[@]}" | xargs cp --parents --target-directory="$tmpdir/b" )
revert_patch "$patch" --recount
# Diff 'a' and 'b' to make a clean patch
( cd "$tmpdir" && git diff --no-index --no-prefix a b > "$patch" ) || true
}
# Copy the patches to a temporary directory, fix their lines so as not to
# affect the __LINE__ macro for otherwise unchanged functions further down the
# file, and update $PATCHES to point to the fixed patches.
fix_patches() {
local idx
local i
rm -f "$TMP_DIR"/*.patch
idx=0001
for i in "${!PATCHES[@]}"; do
local old_patch="${PATCHES[$i]}"
local tmp_patch="$TMP_DIR/tmp.patch"
local patch="${PATCHES[$i]}"
local new_patch
new_patch="$TMP_DIR/$idx-fixed-$(basename "$patch")"
cp -f "$old_patch" "$tmp_patch"
refresh_patch "$tmp_patch"
"$FIX_PATCH_LINES" "$tmp_patch" > "$new_patch"
refresh_patch "$new_patch"
PATCHES[i]="$new_patch"
rm -f "$tmp_patch"
idx=$(printf "%04d" $(( 10#$idx + 1 )))
done
}
clean_kernel() {
local cmd=()
cmd=("make")
cmd+=("--silent")
cmd+=("-j$JOBS")
cmd+=("clean")
(
cd "$SRC"
"${cmd[@]}"
)
}
build_kernel() {
local log="$TMP_DIR/build.log"
local objtool_args=()
local cmd=()
objtool_args=("--checksum")
cmd=("make")
# When a patch to a kernel module references a newly created unexported
# symbol which lives in vmlinux or another kernel module, the patched
# kernel build fails with the following error:
#
# ERROR: modpost: "klp_string" [fs/xfs/xfs.ko] undefined!
#
# The undefined symbols are working as designed in that case. They get
# resolved later when the livepatch module build link pulls all the
# disparate objects together into the same kernel module.
#
# It would be good to have a way to tell modpost to skip checking for
# undefined symbols altogether. For now, just convert the error to a
# warning with KBUILD_MODPOST_WARN, and grep out the warning to avoid
# confusing the user.
#
cmd+=("KBUILD_MODPOST_WARN=1")
cmd+=("$VERBOSE")
cmd+=("-j$JOBS")
cmd+=("KCFLAGS=-ffunction-sections -fdata-sections")
cmd+=("OBJTOOL_ARGS=${objtool_args[*]}")
cmd+=("vmlinux")
cmd+=("modules")
(
cd "$SRC"
"${cmd[@]}" \
1> >(tee -a "$log") \
2> >(tee -a "$log" | grep0 -v "modpost.*undefined!" >&2)
)
}
find_objects() {
local opts=("$@")
# Find root-level vmlinux.o and non-root-level .ko files,
# excluding klp-tmp/ and .git/
find "$OBJ" \( -path "$TMP_DIR" -o -path "$OBJ/.git" -o -regex "$OBJ/[^/][^/]*\.ko" \) -prune -o \
-type f "${opts[@]}" \
\( -name "*.ko" -o -path "$OBJ/vmlinux.o" \) \
-printf '%P\n'
}
# Copy all .o archives to $ORIG_DIR
copy_orig_objects() {
local files=()
rm -rf "$ORIG_DIR"
mkdir -p "$ORIG_DIR"
find_objects | mapfile -t files
xtrace_save "copying orig objects"
for _file in "${files[@]}"; do
local rel_file="${_file/.ko/.o}"
local file="$OBJ/$rel_file"
local file_dir="$(dirname "$file")"
local orig_file="$ORIG_DIR/$rel_file"
local orig_dir="$(dirname "$orig_file")"
local cmd_file="$file_dir/.$(basename "$file").cmd"
[[ ! -f "$file" ]] && die "missing $(basename "$file") for $_file"
mkdir -p "$orig_dir"
cp -f "$file" "$orig_dir"
[[ -e "$cmd_file" ]] && cp -f "$cmd_file" "$orig_dir"
done
xtrace_restore
mv -f "$TMP_DIR/build.log" "$ORIG_DIR"
touch "$TIMESTAMP"
}
# Copy all changed objects to $PATCHED_DIR
copy_patched_objects() {
local files=()
local opts=()
local found=0
rm -rf "$PATCHED_DIR"
mkdir -p "$PATCHED_DIR"
# Note this doesn't work with some configs, thus the 'cmp' below.
opts=("-newer")
opts+=("$TIMESTAMP")
find_objects "${opts[@]}" | mapfile -t files
xtrace_save "copying changed objects"
for _file in "${files[@]}"; do
local rel_file="${_file/.ko/.o}"
local file="$OBJ/$rel_file"
local orig_file="$ORIG_DIR/$rel_file"
local patched_file="$PATCHED_DIR/$rel_file"
local patched_dir="$(dirname "$patched_file")"
[[ ! -f "$file" ]] && die "missing $(basename "$file") for $_file"
cmp -s "$orig_file" "$file" && continue
mkdir -p "$patched_dir"
cp -f "$file" "$patched_dir"
found=1
done
xtrace_restore
(( found == 0 )) && die "no changes detected"
mv -f "$TMP_DIR/build.log" "$PATCHED_DIR"
}
# Diff changed objects, writing output object to $DIFF_DIR
diff_objects() {
local log="$KLP_DIFF_LOG"
local files=()
local opts=()
rm -rf "$DIFF_DIR"
mkdir -p "$DIFF_DIR"
find "$PATCHED_DIR" -type f -name "*.o" | mapfile -t files
[[ ${#files[@]} -eq 0 ]] && die "no changes detected"
[[ -v DEBUG_CLONE ]] && opts=("--debug")
# Diff all changed objects
for file in "${files[@]}"; do
local rel_file="${file#"$PATCHED_DIR"/}"
local orig_file="$rel_file"
local patched_file="$PATCHED_DIR/$rel_file"
local out_file="$DIFF_DIR/$rel_file"
local filter=()
local cmd=()
mkdir -p "$(dirname "$out_file")"
cmd=("$SRC/tools/objtool/objtool")
cmd+=("klp")
cmd+=("diff")
(( ${#opts[@]} > 0 )) && cmd+=("${opts[@]}")
cmd+=("$orig_file")
cmd+=("$patched_file")
cmd+=("$out_file")
if [[ -v DIFF_CHECKSUM ]]; then
filter=("grep0")
filter+=("-Ev")
filter+=("DEBUG: .*checksum: ")
else
filter=("cat")
fi
(
cd "$ORIG_DIR"
"${cmd[@]}" \
1> >(tee -a "$log") \
2> >(tee -a "$log" | "${filter[@]}" >&2) || \
die "objtool klp diff failed"
)
done
}
# For each changed object, run objtool with --debug-checksum to get the
# per-instruction checksums, and then diff those to find the first changed
# instruction for each function.
diff_checksums() {
local orig_log="$ORIG_DIR/checksum.log"
local patched_log="$PATCHED_DIR/checksum.log"
local -A funcs
local cmd=()
local line
local file
local func
gawk '/\.o: changed function: / {
sub(/:$/, "", $1)
print $1, $NF
}' "$KLP_DIFF_LOG" | mapfile -t lines
for line in "${lines[@]}"; do
read -r file func <<< "$line"
if [[ ! -v funcs["$file"] ]]; then
funcs["$file"]="$func"
else
funcs["$file"]+=" $func"
fi
done
cmd=("$SRC/tools/objtool/objtool")
cmd+=("--checksum")
cmd+=("--link")
cmd+=("--dry-run")
for file in "${!funcs[@]}"; do
local opt="--debug-checksum=${funcs[$file]// /,}"
(
cd "$ORIG_DIR"
"${cmd[@]}" "$opt" "$file" &> "$orig_log" || \
( cat "$orig_log" >&2; die "objtool --debug-checksum failed" )
cd "$PATCHED_DIR"
"${cmd[@]}" "$opt" "$file" &> "$patched_log" || \
( cat "$patched_log" >&2; die "objtool --debug-checksum failed" )
)
for func in ${funcs[$file]}; do
diff <( grep0 -E "^DEBUG: .*checksum: $func " "$orig_log" | sed "s|$ORIG_DIR/||") \
<( grep0 -E "^DEBUG: .*checksum: $func " "$patched_log" | sed "s|$PATCHED_DIR/||") \
| gawk '/^< DEBUG: / {
gsub(/:/, "")
printf "%s: %s: %s\n", $3, $5, $6
exit
}' || true
done
done
}
# Build and post-process livepatch module in $KMOD_DIR
build_patch_module() {
local makefile="$KMOD_DIR/Kbuild"
local log="$KMOD_DIR/build.log"
local kmod_file
local cflags=()
local files=()
local cmd=()
rm -rf "$KMOD_DIR"
mkdir -p "$KMOD_DIR"
cp -f "$SRC/scripts/livepatch/init.c" "$KMOD_DIR"
echo "obj-m := $NAME.o" > "$makefile"
echo -n "$NAME-y := init.o" >> "$makefile"
find "$DIFF_DIR" -type f -name "*.o" | mapfile -t files
[[ ${#files[@]} -eq 0 ]] && die "no changes detected"
for file in "${files[@]}"; do
local rel_file="${file#"$DIFF_DIR"/}"
local orig_file="$ORIG_DIR/$rel_file"
local orig_dir="$(dirname "$orig_file")"
local kmod_file="$KMOD_DIR/$rel_file"
local kmod_dir="$(dirname "$kmod_file")"
local cmd_file="$orig_dir/.$(basename "$file").cmd"
mkdir -p "$kmod_dir"
cp -f "$file" "$kmod_dir"
[[ -e "$cmd_file" ]] && cp -f "$cmd_file" "$kmod_dir"
# Tell kbuild this is a prebuilt object
cp -f "$file" "${kmod_file}_shipped"
echo -n " $rel_file" >> "$makefile"
done
echo >> "$makefile"
cflags=("-ffunction-sections")
cflags+=("-fdata-sections")
[[ $REPLACE -eq 0 ]] && cflags+=("-DKLP_NO_REPLACE")
cmd=("make")
cmd+=("$VERBOSE")
cmd+=("-j$JOBS")
cmd+=("--directory=.")
cmd+=("M=$KMOD_DIR")
cmd+=("KCFLAGS=${cflags[*]}")
# Build a "normal" kernel module with init.c and the diffed objects
(
cd "$SRC"
"${cmd[@]}" \
1> >(tee -a "$log") \
2> >(tee -a "$log" >&2)
)
kmod_file="$KMOD_DIR/$NAME.ko"
# Save off the intermediate binary for debugging
cp -f "$kmod_file" "$kmod_file.orig"
# Work around issue where slight .config change makes corrupt BTF
objcopy --remove-section=.BTF "$kmod_file"
# Fix (and work around) linker wreckage for klp syms / relocs
"$SRC/tools/objtool/objtool" klp post-link "$kmod_file" || die "objtool klp post-link failed"
cp -f "$kmod_file" "$OUTFILE"
}
################################################################################
process_args "$@"
do_init
if (( SHORT_CIRCUIT <= 1 )); then
status "Validating patch(es)"
validate_patches
status "Building original kernel"
clean_kernel
build_kernel
status "Copying original object files"
copy_orig_objects
fi
if (( SHORT_CIRCUIT <= 2 )); then
status "Fixing patch(es)"
fix_patches
apply_patches
status "Building patched kernel"
build_kernel
revert_patches
status "Copying patched object files"
copy_patched_objects
fi
if (( SHORT_CIRCUIT <= 3 )); then
status "Diffing objects"
diff_objects
if [[ -v DIFF_CHECKSUM ]]; then
status "Finding first changed instructions"
diff_checksums
fi
fi
if (( SHORT_CIRCUIT <= 4 )); then
status "Building patch module: $OUTFILE"
build_patch_module
fi
status "SUCCESS"

5
scripts/mod/modpost.c
View File

@@ -606,6 +606,11 @@ static int ignore_undef_symbol(struct elf_info *info, const char *symname)
strstarts(symname, "_savevr_") ||
strcmp(symname, ".TOC.") == 0)
return 1;
/* ignore linker-created section bounds variables */
if (strstarts(symname, "__start_") || strstarts(symname, "__stop_"))
return 1;
/* Do not ignore this symbol */
return 0;
}

22
scripts/module.lds.S
View File

@@ -34,16 +34,22 @@ SECTIONS {
__patchable_function_entries : { *(__patchable_function_entries) }
__klp_funcs 0: ALIGN(8) { KEEP(*(__klp_funcs)) }
__klp_objects 0: ALIGN(8) {
__start_klp_objects = .;
KEEP(*(__klp_objects))
__stop_klp_objects = .;
}
#ifdef CONFIG_ARCH_USES_CFI_TRAPS
__kcfi_traps : { KEEP(*(.kcfi_traps)) }
__kcfi_traps : { KEEP(*(.kcfi_traps)) }
#endif
#ifdef CONFIG_LTO_CLANG
/*
* With CONFIG_LTO_CLANG, LLD always enables -fdata-sections and
* -ffunction-sections, which increases the size of the final module.
* Merge the split sections in the final binary.
*/
.text : {
*(.text .text.[0-9a-zA-Z_]*)
}
.bss : {
*(.bss .bss.[0-9a-zA-Z_]*)
*(.bss..L*)
@@ -58,7 +64,7 @@ SECTIONS {
*(.rodata .rodata.[0-9a-zA-Z_]*)
*(.rodata..L*)
}
#endif
MOD_SEPARATE_CODETAG_SECTIONS()
}

34
tools/arch/x86/tools/gen-cpu-feature-names-x86.awk Normal file
View File

@@ -0,0 +1,34 @@
#!/bin/awk -f
# SPDX-License-Identifier: GPL-2.0
#
# Copyright (c) 2025, Oracle and/or its affiliates.
#
# Usage: awk -f gen-cpu-feature-names-x86.awk cpufeatures.h > cpu-feature-names.c
#
BEGIN {
print "/* cpu feature name array generated from cpufeatures.h */"
print "/* Do not change this code. */"
print
print "static const char *cpu_feature_names[(NCAPINTS+NBUGINTS)*32] = {"
value_expr = "\\([0-9*+ ]+\\)"
}
/^#define X86_FEATURE_/ {
if (match($0, value_expr)) {
value = substr($0, RSTART + 1, RLENGTH - 2)
print "\t[" value "] = \"" $2 "\","
}
}
/^#define X86_BUG_/ {
if (match($0, value_expr)) {
value = substr($0, RSTART + 1, RLENGTH - 2)
print "\t[NCAPINTS*32+(" value ")] = \"" $2 "\","
}
}
END {
print "};"
}

2
tools/build/Build Normal file
View File

@@ -0,0 +1,2 @@
hostprogs := fixdep
fixdep-y := fixdep.o

21
tools/build/Makefile
View File

@@ -37,5 +37,22 @@ ifneq ($(wildcard $(TMP_O)),)
$(Q)$(MAKE) -C feature OUTPUT=$(TMP_O) clean >/dev/null
endif
$(OUTPUT)fixdep: $(srctree)/tools/build/fixdep.c
$(QUIET_CC)$(HOSTCC) $(KBUILD_HOSTCFLAGS) $(KBUILD_HOSTLDFLAGS) -o $@ $<
FIXDEP := $(OUTPUT)fixdep
FIXDEP_IN := $(OUTPUT)fixdep-in.o
# To track fixdep's dependencies properly, fixdep needs to run on itself.
# Build it twice the first time.
$(FIXDEP_IN): FORCE
$(Q)if [ ! -f $(FIXDEP) ]; then \
$(MAKE) $(build)=fixdep HOSTCFLAGS="$(KBUILD_HOSTCFLAGS)"; \
rm -f $(FIXDEP).o; \
fi
$(Q)$(MAKE) $(build)=fixdep HOSTCFLAGS="$(KBUILD_HOSTCFLAGS)"
$(FIXDEP): $(FIXDEP_IN)
$(QUIET_LINK)$(HOSTCC) $(FIXDEP_IN) $(KBUILD_HOSTLDFLAGS) -o $@
FORCE:
.PHONY: FORCE

4
tools/build/Makefile.feature
View File

@@ -315,5 +315,7 @@ endef
ifeq ($(FEATURE_DISPLAY_DEFERRED),)
$(call feature_display_entries)
$(info )
ifeq ($(feature_display),1)
$(info )
endif
endif

10
tools/include/linux/interval_tree_generic.h
View File

@@ -77,7 +77,7 @@ ITSTATIC void ITPREFIX ## _remove(ITSTRUCT *node, \
* Cond2: start <= ITLAST(node) \
*/ \
\
static ITSTRUCT * \
ITSTATIC ITSTRUCT * \
ITPREFIX ## _subtree_search(ITSTRUCT *node, ITTYPE start, ITTYPE last) \
{ \
while (true) { \
@@ -104,12 +104,8 @@ ITPREFIX ## _subtree_search(ITSTRUCT *node, ITTYPE start, ITTYPE last) \
if (ITSTART(node) <= last) { /* Cond1 */ \
if (start <= ITLAST(node)) /* Cond2 */ \
return node; /* node is leftmost match */ \
if (node->ITRB.rb_right) { \
node = rb_entry(node->ITRB.rb_right, \
ITSTRUCT, ITRB); \
if (start <= node->ITSUBTREE) \
continue; \
} \
node = rb_entry(node->ITRB.rb_right, ITSTRUCT, ITRB); \
continue; \
} \
return NULL; /* No match */ \
} \

76
tools/include/linux/livepatch_external.h Normal file
View File

@@ -0,0 +1,76 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* External livepatch interfaces for patch creation tooling
*/
#ifndef _LINUX_LIVEPATCH_EXTERNAL_H_
#define _LINUX_LIVEPATCH_EXTERNAL_H_
#include <linux/types.h>
#define KLP_RELOC_SEC_PREFIX ".klp.rela."
#define KLP_SYM_PREFIX ".klp.sym."
#define __KLP_PRE_PATCH_PREFIX __klp_pre_patch_callback_
#define __KLP_POST_PATCH_PREFIX __klp_post_patch_callback_
#define __KLP_PRE_UNPATCH_PREFIX __klp_pre_unpatch_callback_
#define __KLP_POST_UNPATCH_PREFIX __klp_post_unpatch_callback_
#define KLP_PRE_PATCH_PREFIX __stringify(__KLP_PRE_PATCH_PREFIX)
#define KLP_POST_PATCH_PREFIX __stringify(__KLP_POST_PATCH_PREFIX)
#define KLP_PRE_UNPATCH_PREFIX __stringify(__KLP_PRE_UNPATCH_PREFIX)
#define KLP_POST_UNPATCH_PREFIX __stringify(__KLP_POST_UNPATCH_PREFIX)
struct klp_object;
typedef int (*klp_pre_patch_t)(struct klp_object *obj);
typedef void (*klp_post_patch_t)(struct klp_object *obj);
typedef void (*klp_pre_unpatch_t)(struct klp_object *obj);
typedef void (*klp_post_unpatch_t)(struct klp_object *obj);
/**
* struct klp_callbacks - pre/post live-(un)patch callback structure
* @pre_patch: executed before code patching
* @post_patch: executed after code patching
* @pre_unpatch: executed before code unpatching
* @post_unpatch: executed after code unpatching
* @post_unpatch_enabled: flag indicating if post-unpatch callback
* should run
*
* All callbacks are optional. Only the pre-patch callback, if provided,
* will be unconditionally executed. If the parent klp_object fails to
* patch for any reason, including a non-zero error status returned from
* the pre-patch callback, no further callbacks will be executed.
*/
struct klp_callbacks {
klp_pre_patch_t pre_patch;
klp_post_patch_t post_patch;
klp_pre_unpatch_t pre_unpatch;
klp_post_unpatch_t post_unpatch;
bool post_unpatch_enabled;
};
/*
* 'struct klp_{func,object}_ext' are compact "external" representations of
* 'struct klp_{func,object}'. They are used by objtool for livepatch
* generation. The structs are then read by the livepatch module and converted
* to the real structs before calling klp_enable_patch().
*
* TODO make these the official API for klp_enable_patch(). That should
* simplify livepatch's interface as well as its data structure lifetime
* management.
*/
struct klp_func_ext {
const char *old_name;
void *new_func;
unsigned long sympos;
};
struct klp_object_ext {
const char *name;
struct klp_func_ext *funcs;
struct klp_callbacks callbacks;
unsigned int nr_funcs;
};
#endif /* _LINUX_LIVEPATCH_EXTERNAL_H_ */

2
tools/include/linux/objtool_types.h
View File

@@ -67,4 +67,6 @@ struct unwind_hint {
#define ANNOTYPE_REACHABLE 8
#define ANNOTYPE_NOCFI 9
#define ANNOTYPE_DATA_SPECIAL 1
#endif /* _LINUX_OBJTOOL_TYPES_H */

14
tools/include/linux/string.h
View File

@@ -44,6 +44,20 @@ static inline bool strstarts(const char *str, const char *prefix)
return strncmp(str, prefix, strlen(prefix)) == 0;
}
/*
* Checks if a string ends with another.
*/
static inline bool str_ends_with(const char *str, const char *substr)
{
size_t len = strlen(str);
size_t sublen = strlen(substr);
if (sublen > len)
return false;
return !strcmp(str + len - sublen, substr);
}
extern char * __must_check skip_spaces(const char *);
extern char *strim(char *);

3
tools/objtool/.gitignore vendored
View File

@@ -1,5 +1,8 @@
# SPDX-License-Identifier: GPL-2.0-only
arch/x86/lib/cpu-feature-names.c
arch/x86/lib/inat-tables.c
/objtool
feature
FEATURE-DUMP.objtool
fixdep
libsubcmd/

7
tools/objtool/Build
View File

@@ -8,8 +8,11 @@ objtool-y += builtin-check.o
objtool-y += elf.o
objtool-y += objtool.o
objtool-$(BUILD_ORC) += orc_gen.o
objtool-$(BUILD_ORC) += orc_dump.o
objtool-$(BUILD_DISAS) += disas.o
objtool-$(BUILD_DISAS) += trace.o
objtool-$(BUILD_ORC) += orc_gen.o orc_dump.o
objtool-$(BUILD_KLP) += builtin-klp.o klp-diff.o klp-post-link.o
objtool-y += libstring.o
objtool-y += libctype.o

74
tools/objtool/Makefile
View File

@@ -2,6 +2,28 @@
include ../scripts/Makefile.include
include ../scripts/Makefile.arch
ifeq ($(SRCARCH),x86)
BUILD_ORC := y
ARCH_HAS_KLP := y
endif
ifeq ($(SRCARCH),loongarch)
BUILD_ORC := y
endif
ifeq ($(ARCH_HAS_KLP),y)
HAVE_XXHASH = $(shell printf "$(pound)include <xxhash.h>\nXXH3_state_t *state;int main() {}" | \
$(HOSTCC) -xc - -o /dev/null -lxxhash 2> /dev/null && echo y || echo n)
ifeq ($(HAVE_XXHASH),y)
BUILD_KLP := y
LIBXXHASH_CFLAGS := $(shell $(HOSTPKG_CONFIG) libxxhash --cflags 2>/dev/null) \
-DBUILD_KLP
LIBXXHASH_LIBS := $(shell $(HOSTPKG_CONFIG) libxxhash --libs 2>/dev/null || echo -lxxhash)
endif
endif
export BUILD_ORC BUILD_KLP
ifeq ($(srctree),)
srctree := $(patsubst %/,%,$(dir $(CURDIR)))
srctree := $(patsubst %/,%,$(dir $(srctree)))
@@ -23,6 +45,11 @@ LIBELF_LIBS := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lel
all: $(OBJTOOL)
WARNINGS := -Werror -Wall -Wextra -Wmissing-prototypes \
-Wmissing-declarations -Wwrite-strings \
-Wno-implicit-fallthrough -Wno-sign-compare \
-Wno-unused-parameter
INCLUDES := -I$(srctree)/tools/include \
-I$(srctree)/tools/include/uapi \
-I$(srctree)/tools/arch/$(HOSTARCH)/include/uapi \
@@ -30,11 +57,11 @@ INCLUDES := -I$(srctree)/tools/include \
-I$(srctree)/tools/objtool/include \
-I$(srctree)/tools/objtool/arch/$(SRCARCH)/include \
-I$(LIBSUBCMD_OUTPUT)/include
# Note, EXTRA_WARNINGS here was determined for CC and not HOSTCC, it
# is passed here to match a legacy behavior.
WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed -Wno-nested-externs
OBJTOOL_CFLAGS := -Werror $(WARNINGS) $(KBUILD_HOSTCFLAGS) -g $(INCLUDES) $(LIBELF_FLAGS)
OBJTOOL_LDFLAGS := $(LIBELF_LIBS) $(LIBSUBCMD) $(KBUILD_HOSTLDFLAGS)
OBJTOOL_CFLAGS := -std=gnu11 -fomit-frame-pointer -O2 -g $(WARNINGS) \
$(INCLUDES) $(LIBELF_FLAGS) $(LIBXXHASH_CFLAGS) $(HOSTCFLAGS)
OBJTOOL_LDFLAGS := $(LIBSUBCMD) $(LIBELF_LIBS) $(LIBXXHASH_LIBS) $(HOSTLDFLAGS)
# Allow old libelf to be used:
elfshdr := $(shell echo '$(pound)include <libelf.h>' | $(HOSTCC) $(OBJTOOL_CFLAGS) -x c -E - 2>/dev/null | grep elf_getshdr)
@@ -43,20 +70,32 @@ OBJTOOL_CFLAGS += $(if $(elfshdr),,-DLIBELF_USE_DEPRECATED)
# Always want host compilation.
HOST_OVERRIDES := CC="$(HOSTCC)" LD="$(HOSTLD)" AR="$(HOSTAR)"
#
# To support disassembly, objtool needs libopcodes which is provided
# with libbdf (binutils-dev or binutils-devel package).
#
FEATURE_USER = .objtool
FEATURE_TESTS = libbfd disassembler-init-styled
FEATURE_DISPLAY =
include $(srctree)/tools/build/Makefile.feature
ifeq ($(feature-disassembler-init-styled), 1)
OBJTOOL_CFLAGS += -DDISASM_INIT_STYLED
endif
BUILD_DISAS := n
ifeq ($(feature-libbfd),1)
BUILD_DISAS := y
OBJTOOL_CFLAGS += -DDISAS -DPACKAGE="objtool"
OBJTOOL_LDFLAGS += -lopcodes
endif
export BUILD_DISAS
AWK = awk
MKDIR = mkdir
BUILD_ORC := n
ifeq ($(SRCARCH),x86)
BUILD_ORC := y
endif
ifeq ($(SRCARCH),loongarch)
BUILD_ORC := y
endif
export BUILD_ORC
export srctree OUTPUT CFLAGS SRCARCH AWK
include $(srctree)/tools/build/Makefile.include
@@ -86,7 +125,10 @@ $(LIBSUBCMD)-clean:
clean: $(LIBSUBCMD)-clean
$(call QUIET_CLEAN, objtool) $(RM) $(OBJTOOL)
$(Q)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
$(Q)$(RM) $(OUTPUT)arch/x86/lib/cpu-feature-names.c $(OUTPUT)fixdep
$(Q)$(RM) $(OUTPUT)arch/x86/lib/inat-tables.c $(OUTPUT)fixdep
$(Q)$(RM) -- $(OUTPUT)FEATURE-DUMP.objtool
$(Q)$(RM) -r -- $(OUTPUT)feature
FORCE:

29
tools/objtool/arch/loongarch/decode.c
View File

@@ -1,13 +1,25 @@
// SPDX-License-Identifier: GPL-2.0-or-later
#include <string.h>
#include <objtool/check.h>
#include <objtool/disas.h>
#include <objtool/warn.h>
#include <asm/inst.h>
#include <asm/orc_types.h>
#include <linux/objtool_types.h>
#include <arch/elf.h>
int arch_ftrace_match(char *name)
const char *arch_reg_name[CFI_NUM_REGS] = {
"zero", "ra", "tp", "sp",
"a0", "a1", "a2", "a3",
"a4", "a5", "a6", "a7",
"t0", "t1", "t2", "t3",
"t4", "t5", "t6", "t7",
"t8", "u0", "fp", "s0",
"s1", "s2", "s3", "s4",
"s5", "s6", "s7", "s8"
};
int arch_ftrace_match(const char *name)
{
return !strcmp(name, "_mcount");
}
@@ -17,9 +29,9 @@ unsigned long arch_jump_destination(struct instruction *insn)
return insn->offset + (insn->immediate << 2);
}
unsigned long arch_dest_reloc_offset(int addend)
s64 arch_insn_adjusted_addend(struct instruction *insn, struct reloc *reloc)
{
return addend;
return reloc_addend(reloc);
}
bool arch_pc_relative_reloc(struct reloc *reloc)
@@ -414,3 +426,14 @@ unsigned long arch_jump_table_sym_offset(struct reloc *reloc, struct reloc *tabl
return reloc->sym->offset + reloc_addend(reloc);
}
}
#ifdef DISAS
int arch_disas_info_init(struct disassemble_info *dinfo)
{
return disas_info_init(dinfo, bfd_arch_loongarch,
bfd_mach_loongarch32, bfd_mach_loongarch64,
NULL);
}
#endif /* DISAS */

1
tools/objtool/arch/loongarch/orc.c
View File

@@ -5,7 +5,6 @@
#include <objtool/check.h>
#include <objtool/orc.h>
#include <objtool/warn.h>
#include <objtool/endianness.h>
int init_orc_entry(struct orc_entry *orc, struct cfi_state *cfi, struct instruction *insn)
{

5
tools/objtool/arch/loongarch/special.c
View File

@@ -194,3 +194,8 @@ struct reloc *arch_find_switch_table(struct objtool_file *file,
return rodata_reloc;
}
const char *arch_cpu_feature_name(int feature_number)
{
return NULL;
}

31
tools/objtool/arch/powerpc/decode.c
View File

@@ -3,20 +3,32 @@
#include <stdio.h>
#include <stdlib.h>
#include <objtool/check.h>
#include <objtool/disas.h>
#include <objtool/elf.h>
#include <objtool/arch.h>
#include <objtool/warn.h>
#include <objtool/builtin.h>
#include <objtool/endianness.h>
int arch_ftrace_match(char *name)
const char *arch_reg_name[CFI_NUM_REGS] = {
"r0", "sp", "r2", "r3",
"r4", "r5", "r6", "r7",
"r8", "r9", "r10", "r11",
"r12", "r13", "r14", "r15",
"r16", "r17", "r18", "r19",
"r20", "r21", "r22", "r23",
"r24", "r25", "r26", "r27",
"r28", "r29", "r30", "r31",
"ra"
};
int arch_ftrace_match(const char *name)
{
return !strcmp(name, "_mcount");
}
unsigned long arch_dest_reloc_offset(int addend)
s64 arch_insn_adjusted_addend(struct instruction *insn, struct reloc *reloc)
{
return addend;
return reloc_addend(reloc);
}
bool arch_callee_saved_reg(unsigned char reg)
@@ -128,3 +140,14 @@ unsigned int arch_reloc_size(struct reloc *reloc)
return 8;
}
}
#ifdef DISAS
int arch_disas_info_init(struct disassemble_info *dinfo)
{
return disas_info_init(dinfo, bfd_arch_powerpc,
bfd_mach_ppc, bfd_mach_ppc64,
NULL);
}
#endif /* DISAS */

5
tools/objtool/arch/powerpc/special.c
View File

@@ -18,3 +18,8 @@ struct reloc *arch_find_switch_table(struct objtool_file *file,
{
exit(-1);
}
const char *arch_cpu_feature_name(int feature_number)
{
return NULL;
}

13
tools/objtool/arch/x86/Build
View File

@@ -1,5 +1,5 @@
objtool-y += special.o
objtool-y += decode.o
objtool-y += special.o
objtool-y += orc.o
inat_tables_script = ../arch/x86/tools/gen-insn-attr-x86.awk
@@ -12,3 +12,14 @@ $(OUTPUT)arch/x86/lib/inat-tables.c: $(inat_tables_script) $(inat_tables_maps)
$(OUTPUT)arch/x86/decode.o: $(OUTPUT)arch/x86/lib/inat-tables.c
CFLAGS_decode.o += -I$(OUTPUT)arch/x86/lib
cpu_features = ../arch/x86/include/asm/cpufeatures.h
cpu_features_script = ../arch/x86/tools/gen-cpu-feature-names-x86.awk
$(OUTPUT)arch/x86/lib/cpu-feature-names.c: $(cpu_features_script) $(cpu_features)
$(call rule_mkdir)
$(Q)$(call echo-cmd,gen)$(AWK) -f $(cpu_features_script) $(cpu_features) > $@
$(OUTPUT)arch/x86/special.o: $(OUTPUT)arch/x86/lib/cpu-feature-names.c
CFLAGS_special.o += -I$(OUTPUT)arch/x86/lib

111
tools/objtool/arch/x86/decode.c
View File

@@ -16,14 +16,22 @@
#include <asm/orc_types.h>
#include <objtool/check.h>
#include <objtool/disas.h>
#include <objtool/elf.h>
#include <objtool/arch.h>
#include <objtool/warn.h>
#include <objtool/endianness.h>
#include <objtool/builtin.h>
#include <arch/elf.h>
int arch_ftrace_match(char *name)
const char *arch_reg_name[CFI_NUM_REGS] = {
"rax", "rcx", "rdx", "rbx",
"rsp", "rbp", "rsi", "rdi",
"r8", "r9", "r10", "r11",
"r12", "r13", "r14", "r15",
"ra"
};
int arch_ftrace_match(const char *name)
{
return !strcmp(name, "__fentry__");
}
@@ -68,9 +76,65 @@ bool arch_callee_saved_reg(unsigned char reg)
}
}
unsigned long arch_dest_reloc_offset(int addend)
/* Undo the effects of __pa_symbol() if necessary */
static unsigned long phys_to_virt(unsigned long pa)
{
return addend + 4;
s64 va = pa;
if (va > 0)
va &= ~(0x80000000);
return va;
}
s64 arch_insn_adjusted_addend(struct instruction *insn, struct reloc *reloc)
{
s64 addend = reloc_addend(reloc);
if (arch_pc_relative_reloc(reloc))
addend += insn->offset + insn->len - reloc_offset(reloc);
return phys_to_virt(addend);
}
static void scan_for_insn(struct section *sec, unsigned long offset,
unsigned long *insn_off, unsigned int *insn_len)
{
unsigned long o = 0;
struct insn insn;
while (1) {
insn_decode(&insn, sec->data->d_buf + o, sec_size(sec) - o,
INSN_MODE_64);
if (o + insn.length > offset) {
*insn_off = o;
*insn_len = insn.length;
return;
}
o += insn.length;
}
}
u64 arch_adjusted_addend(struct reloc *reloc)
{
unsigned int type = reloc_type(reloc);
s64 addend = reloc_addend(reloc);
unsigned long insn_off;
unsigned int insn_len;
if (type == R_X86_64_PLT32)
return addend + 4;
if (type != R_X86_64_PC32 || !is_text_sec(reloc->sec->base))
return addend;
scan_for_insn(reloc->sec->base, reloc_offset(reloc),
&insn_off, &insn_len);
return addend + insn_off + insn_len - reloc_offset(reloc);
}
unsigned long arch_jump_destination(struct instruction *insn)
@@ -189,15 +253,6 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
op2 = ins.opcode.bytes[1];
op3 = ins.opcode.bytes[2];
/*
* XXX hack, decoder is buggered and thinks 0xea is 7 bytes long.
*/
if (op1 == 0xea) {
insn->len = 1;
insn->type = INSN_BUG;
return 0;
}
if (ins.rex_prefix.nbytes) {
rex = ins.rex_prefix.bytes[0];
rex_w = X86_REX_W(rex) >> 3;
@@ -503,6 +558,12 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
break;
case 0x90:
if (rex_b) /* XCHG %r8, %rax */
break;
if (prefix == 0xf3) /* REP NOP := PAUSE */
break;
insn->type = INSN_NOP;
break;
@@ -556,13 +617,14 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
} else if (op2 == 0x0b || op2 == 0xb9) {
/* ud2 */
/* ud2, ud1 */
insn->type = INSN_BUG;
} else if (op2 == 0x0d || op2 == 0x1f) {
} else if (op2 == 0x1f) {
/* nopl/nopw */
insn->type = INSN_NOP;
/* 0f 1f /0 := NOPL */
if (modrm_reg == 0)
insn->type = INSN_NOP;
} else if (op2 == 0x1e) {
@@ -692,6 +754,10 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec
insn->type = INSN_SYSRET;
break;
case 0xd6: /* udb */
insn->type = INSN_BUG;
break;
case 0xe0: /* loopne */
case 0xe1: /* loope */
case 0xe2: /* loop */
@@ -892,3 +958,14 @@ bool arch_absolute_reloc(struct elf *elf, struct reloc *reloc)
return false;
}
}
#ifdef DISAS
int arch_disas_info_init(struct disassemble_info *dinfo)
{
return disas_info_init(dinfo, bfd_arch_i386,
bfd_mach_i386_i386, bfd_mach_x86_64,
"att");
}
#endif /* DISAS */

1
tools/objtool/arch/x86/orc.c
View File

@@ -5,7 +5,6 @@
#include <objtool/check.h>
#include <objtool/orc.h>
#include <objtool/warn.h>
#include <objtool/endianness.h>
int init_orc_entry(struct orc_entry *orc, struct cfi_state *cfi, struct instruction *insn)
{

12
tools/objtool/arch/x86/special.c
View File

@@ -4,6 +4,10 @@
#include <objtool/special.h>
#include <objtool/builtin.h>
#include <objtool/warn.h>
#include <asm/cpufeatures.h>
/* cpu feature name array generated from cpufeatures.h */
#include "cpu-feature-names.c"
void arch_handle_alternative(struct special_alt *alt)
{
@@ -89,7 +93,7 @@ struct reloc *arch_find_switch_table(struct objtool_file *file,
/* look for a relocation which references .rodata */
text_reloc = find_reloc_by_dest_range(file->elf, insn->sec,
insn->offset, insn->len);
if (!text_reloc || text_reloc->sym->type != STT_SECTION ||
if (!text_reloc || !is_sec_sym(text_reloc->sym) ||
!text_reloc->sym->sec->rodata)
return NULL;
@@ -134,3 +138,9 @@ struct reloc *arch_find_switch_table(struct objtool_file *file,
*table_size = 0;
return rodata_reloc;
}
const char *arch_cpu_feature_name(int feature_number)
{
return (feature_number < ARRAY_SIZE(cpu_feature_names)) ?
cpu_feature_names[feature_number] : NULL;
}

102
tools/objtool/builtin-check.c
View File

@@ -73,35 +73,41 @@ static int parse_hacks(const struct option *opt, const char *str, int unset)
static const struct option check_options[] = {
OPT_GROUP("Actions:"),
OPT_BOOLEAN(0, "checksum", &opts.checksum, "generate per-function checksums"),
OPT_BOOLEAN(0, "cfi", &opts.cfi, "annotate kernel control flow integrity (kCFI) function preambles"),
OPT_STRING_OPTARG('d', "disas", &opts.disas, "function-pattern", "disassemble functions", "*"),
OPT_CALLBACK_OPTARG('h', "hacks", NULL, NULL, "jump_label,noinstr,skylake", "patch toolchain bugs/limitations", parse_hacks),
OPT_BOOLEAN('i', "ibt", &opts.ibt, "validate and annotate IBT"),
OPT_BOOLEAN('m', "mcount", &opts.mcount, "annotate mcount/fentry calls for ftrace"),
OPT_BOOLEAN('n', "noinstr", &opts.noinstr, "validate noinstr rules"),
OPT_BOOLEAN(0, "orc", &opts.orc, "generate ORC metadata"),
OPT_BOOLEAN('r', "retpoline", &opts.retpoline, "validate and annotate retpoline usage"),
OPT_BOOLEAN(0, "rethunk", &opts.rethunk, "validate and annotate rethunk usage"),
OPT_BOOLEAN(0, "unret", &opts.unret, "validate entry unret placement"),
OPT_INTEGER(0, "prefix", &opts.prefix, "generate prefix symbols"),
OPT_BOOLEAN('l', "sls", &opts.sls, "validate straight-line-speculation mitigations"),
OPT_BOOLEAN('s', "stackval", &opts.stackval, "validate frame pointer rules"),
OPT_BOOLEAN('t', "static-call", &opts.static_call, "annotate static calls"),
OPT_BOOLEAN('u', "uaccess", &opts.uaccess, "validate uaccess rules for SMAP"),
OPT_BOOLEAN(0 , "cfi", &opts.cfi, "annotate kernel control flow integrity (kCFI) function preambles"),
OPT_BOOLEAN(0 , "noabs", &opts.noabs, "reject absolute references in allocatable sections"),
OPT_CALLBACK_OPTARG(0, "dump", NULL, NULL, "orc", "dump metadata", parse_dump),
OPT_BOOLEAN('i', "ibt", &opts.ibt, "validate and annotate IBT"),
OPT_BOOLEAN('m', "mcount", &opts.mcount, "annotate mcount/fentry calls for ftrace"),
OPT_BOOLEAN(0, "noabs", &opts.noabs, "reject absolute references in allocatable sections"),
OPT_BOOLEAN('n', "noinstr", &opts.noinstr, "validate noinstr rules"),
OPT_BOOLEAN(0, "orc", &opts.orc, "generate ORC metadata"),
OPT_BOOLEAN('r', "retpoline", &opts.retpoline, "validate and annotate retpoline usage"),
OPT_BOOLEAN(0, "rethunk", &opts.rethunk, "validate and annotate rethunk usage"),
OPT_BOOLEAN(0, "unret", &opts.unret, "validate entry unret placement"),
OPT_INTEGER(0, "prefix", &opts.prefix, "generate prefix symbols"),
OPT_BOOLEAN('l', "sls", &opts.sls, "validate straight-line-speculation mitigations"),
OPT_BOOLEAN('s', "stackval", &opts.stackval, "validate frame pointer rules"),
OPT_BOOLEAN('t', "static-call", &opts.static_call, "annotate static calls"),
OPT_BOOLEAN('u', "uaccess", &opts.uaccess, "validate uaccess rules for SMAP"),
OPT_CALLBACK_OPTARG(0, "dump", NULL, NULL, "orc", "dump metadata", parse_dump),
OPT_GROUP("Options:"),
OPT_BOOLEAN(0, "backtrace", &opts.backtrace, "unwind on error"),
OPT_BOOLEAN(0, "dry-run", &opts.dryrun, "don't write modifications"),
OPT_BOOLEAN(0, "link", &opts.link, "object is a linked object"),
OPT_BOOLEAN(0, "module", &opts.module, "object is part of a kernel module"),
OPT_BOOLEAN(0, "mnop", &opts.mnop, "nop out mcount call sites"),
OPT_BOOLEAN(0, "no-unreachable", &opts.no_unreachable, "skip 'unreachable instruction' warnings"),
OPT_STRING('o', "output", &opts.output, "file", "output file name"),
OPT_BOOLEAN(0, "sec-address", &opts.sec_address, "print section addresses in warnings"),
OPT_BOOLEAN(0, "stats", &opts.stats, "print statistics"),
OPT_BOOLEAN('v', "verbose", &opts.verbose, "verbose warnings"),
OPT_BOOLEAN(0, "Werror", &opts.werror, "return error on warnings"),
OPT_BOOLEAN(0, "backtrace", &opts.backtrace, "unwind on error"),
OPT_BOOLEAN(0, "backup", &opts.backup, "create backup (.orig) file on warning/error"),
OPT_STRING(0, "debug-checksum", &opts.debug_checksum, "funcs", "enable checksum debug output"),
OPT_BOOLEAN(0, "dry-run", &opts.dryrun, "don't write modifications"),
OPT_BOOLEAN(0, "link", &opts.link, "object is a linked object"),
OPT_BOOLEAN(0, "module", &opts.module, "object is part of a kernel module"),
OPT_BOOLEAN(0, "mnop", &opts.mnop, "nop out mcount call sites"),
OPT_BOOLEAN(0, "no-unreachable", &opts.no_unreachable, "skip 'unreachable instruction' warnings"),
OPT_STRING('o', "output", &opts.output, "file", "output file name"),
OPT_BOOLEAN(0, "sec-address", &opts.sec_address, "print section addresses in warnings"),
OPT_BOOLEAN(0, "stats", &opts.stats, "print statistics"),
OPT_STRING(0, "trace", &opts.trace, "func", "trace function validation"),
OPT_BOOLEAN('v', "verbose", &opts.verbose, "verbose warnings"),
OPT_BOOLEAN(0, "werror", &opts.werror, "return error on warnings"),
OPT_BOOLEAN(0, "wide", &opts.wide, "wide output"),
OPT_END(),
};
@@ -159,7 +165,21 @@ static bool opts_valid(void)
return false;
}
if (opts.hack_jump_label ||
#ifndef BUILD_KLP
if (opts.checksum) {
ERROR("--checksum not supported; install xxhash-devel/libxxhash-dev (version >= 0.8) and recompile");
return false;
}
#endif
if (opts.debug_checksum && !opts.checksum) {
ERROR("--debug-checksum requires --checksum");
return false;
}
if (opts.checksum ||
opts.disas ||
opts.hack_jump_label ||
opts.hack_noinstr ||
opts.ibt ||
opts.mcount ||
@@ -243,15 +263,12 @@ static void save_argv(int argc, const char **argv)
ERROR_GLIBC("strdup(%s)", argv[i]);
exit(1);
}
};
}
}
void print_args(void)
int make_backup(void)
{
char *backup = NULL;
if (opts.output || opts.dryrun)
goto print;
char *backup;
/*
* Make a backup before kbuild deletes the file so the error
@@ -260,33 +277,32 @@ void print_args(void)
backup = malloc(strlen(objname) + strlen(ORIG_SUFFIX) + 1);
if (!backup) {
ERROR_GLIBC("malloc");
goto print;
return 1;
}
strcpy(backup, objname);
strcat(backup, ORIG_SUFFIX);
if (copy_file(objname, backup)) {
backup = NULL;
goto print;
}
if (copy_file(objname, backup))
return 1;
print:
/*
* Print the cmdline args to make it easier to recreate. If '--output'
* wasn't used, add it to the printed args with the backup as input.
* Print the cmdline args to make it easier to recreate.
*/
fprintf(stderr, "%s", orig_argv[0]);
for (int i = 1; i < orig_argc; i++) {
char *arg = orig_argv[i];
if (backup && !strcmp(arg, objname))
/* Modify the printed args to use the backup */
if (!opts.output && !strcmp(arg, objname))
fprintf(stderr, " %s -o %s", backup, objname);
else
fprintf(stderr, " %s", arg);
}
fprintf(stderr, "\n");
return 0;
}
int objtool_run(int argc, const char **argv)
@@ -332,5 +348,5 @@ int objtool_run(int argc, const char **argv)
if (!opts.dryrun && file->elf->changed && elf_write(file->elf))
return 1;
return 0;
return elf_close(file->elf);
}

53
tools/objtool/builtin-klp.c Normal file
View File

@@ -0,0 +1,53 @@
// SPDX-License-Identifier: GPL-2.0-or-later
#include <subcmd/parse-options.h>
#include <string.h>
#include <stdlib.h>
#include <objtool/builtin.h>
#include <objtool/objtool.h>
#include <objtool/klp.h>
struct subcmd {
const char *name;
const char *description;
int (*fn)(int, const char **);
};
static struct subcmd subcmds[] = {
{ "diff", "Generate binary diff of two object files", cmd_klp_diff, },
{ "post-link", "Finalize klp symbols/relocs after module linking", cmd_klp_post_link, },
};
static void cmd_klp_usage(void)
{
fprintf(stderr, "usage: objtool klp <subcommand> [<options>]\n\n");
fprintf(stderr, "Subcommands:\n");
for (int i = 0; i < ARRAY_SIZE(subcmds); i++) {
struct subcmd *cmd = &subcmds[i];
fprintf(stderr, " %s\t%s\n", cmd->name, cmd->description);
}
exit(1);
}
int cmd_klp(int argc, const char **argv)
{
argc--;
argv++;
if (!argc)
cmd_klp_usage();
if (argc) {
for (int i = 0; i < ARRAY_SIZE(subcmds); i++) {
struct subcmd *cmd = &subcmds[i];
if (!strcmp(cmd->name, argv[0]))
return cmd->fn(argc, argv);
}
}
cmd_klp_usage();
return 0;
}

1562
tools/objtool/check.c
View File

File diff suppressed because it is too large Load Diff

1248
tools/objtool/disas.c Normal file
View File

File diff suppressed because it is too large Load Diff

838
tools/objtool/elf.c
View File

File diff suppressed because it is too large Load Diff

16
tools/objtool/include/objtool/arch.h
View File

@@ -71,7 +71,7 @@ struct stack_op {
struct instruction;
int arch_ftrace_match(char *name);
int arch_ftrace_match(const char *name);
void arch_initial_func_cfi_state(struct cfi_init_state *state);
@@ -83,7 +83,8 @@ bool arch_callee_saved_reg(unsigned char reg);
unsigned long arch_jump_destination(struct instruction *insn);
unsigned long arch_dest_reloc_offset(int addend);
s64 arch_insn_adjusted_addend(struct instruction *insn, struct reloc *reloc);
u64 arch_adjusted_addend(struct reloc *reloc);
const char *arch_nop_insn(int len);
const char *arch_ret_insn(int len);
@@ -102,4 +103,15 @@ bool arch_absolute_reloc(struct elf *elf, struct reloc *reloc);
unsigned int arch_reloc_size(struct reloc *reloc);
unsigned long arch_jump_table_sym_offset(struct reloc *reloc, struct reloc *table);
extern const char *arch_reg_name[CFI_NUM_REGS];
#ifdef DISAS
#include <bfd.h>
#include <dis-asm.h>
int arch_disas_info_init(struct disassemble_info *dinfo);
#endif /* DISAS */
#endif /* _ARCH_H */

14
tools/objtool/include/objtool/builtin.h
View File

@@ -9,12 +9,15 @@
struct opts {
/* actions: */
bool cfi;
bool checksum;
bool dump_orc;
bool hack_jump_label;
bool hack_noinstr;
bool hack_skylake;
bool ibt;
bool mcount;
bool noabs;
bool noinstr;
bool orc;
bool retpoline;
@@ -25,11 +28,12 @@ struct opts {
bool static_call;
bool uaccess;
int prefix;
bool cfi;
bool noabs;
const char *disas;
/* options: */
bool backtrace;
bool backup;
const char *debug_checksum;
bool dryrun;
bool link;
bool mnop;
@@ -38,8 +42,10 @@ struct opts {
const char *output;
bool sec_address;
bool stats;
const char *trace;
bool verbose;
bool werror;
bool wide;
};
extern struct opts opts;
@@ -48,6 +54,8 @@ int cmd_parse_options(int argc, const char **argv, const char * const usage[]);
int objtool_run(int argc, const char **argv);
void print_args(void);
int make_backup(void);
int cmd_klp(int argc, const char **argv);
#endif /* _BUILTIN_H */

39
tools/objtool/include/objtool/check.h
View File

@@ -36,6 +36,19 @@ struct alt_group {
struct cfi_state **cfi;
bool ignore;
unsigned int feature;
};
enum alternative_type {
ALT_TYPE_INSTRUCTIONS,
ALT_TYPE_JUMP_TABLE,
ALT_TYPE_EX_TABLE,
};
struct alternative {
struct alternative *next;
struct instruction *insn;
enum alternative_type type;
};
#define INSN_CHUNK_BITS 8
@@ -64,8 +77,11 @@ struct instruction {
noendbr : 1,
unret : 1,
visited : 4,
no_reloc : 1;
/* 10 bit hole */
no_reloc : 1,
hole : 1,
fake : 1,
trace : 1;
/* 9 bit hole */
struct alt_group *alt_group;
struct instruction *jump_dest;
@@ -115,6 +131,15 @@ static inline bool is_jump(struct instruction *insn)
return is_static_jump(insn) || is_dynamic_jump(insn);
}
static inline struct symbol *insn_call_dest(struct instruction *insn)
{
if (insn->type == INSN_JUMP_DYNAMIC ||
insn->type == INSN_CALL_DYNAMIC)
return NULL;
return insn->_call_dest;
}
struct instruction *find_insn(struct objtool_file *file,
struct section *sec, unsigned long offset);
@@ -125,4 +150,14 @@ struct instruction *next_insn_same_sec(struct objtool_file *file, struct instruc
insn && insn->sec == _sec; \
insn = next_insn_same_sec(file, insn))
#define sym_for_each_insn(file, sym, insn) \
for (insn = find_insn(file, sym->sec, sym->offset); \
insn && insn->offset < sym->offset + sym->len; \
insn = next_insn_same_sec(file, insn))
const char *objtool_disas_insn(struct instruction *insn);
extern size_t sym_name_max_len;
extern struct disas_context *objtool_disas_ctx;
#endif /* _CHECK_H */

43
tools/objtool/include/objtool/checksum.h Normal file
View File

@@ -0,0 +1,43 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
#ifndef _OBJTOOL_CHECKSUM_H
#define _OBJTOOL_CHECKSUM_H
#include <objtool/elf.h>
#ifdef BUILD_KLP
static inline void checksum_init(struct symbol *func)
{
if (func && !func->csum.state) {
func->csum.state = XXH3_createState();
XXH3_64bits_reset(func->csum.state);
}
}
static inline void checksum_update(struct symbol *func,
struct instruction *insn,
const void *data, size_t size)
{
XXH3_64bits_update(func->csum.state, data, size);
dbg_checksum(func, insn, XXH3_64bits_digest(func->csum.state));
}
static inline void checksum_finish(struct symbol *func)
{
if (func && func->csum.state) {
func->csum.checksum = XXH3_64bits_digest(func->csum.state);
func->csum.state = NULL;
}
}
#else /* !BUILD_KLP */
static inline void checksum_init(struct symbol *func) {}
static inline void checksum_update(struct symbol *func,
struct instruction *insn,
const void *data, size_t size) {}
static inline void checksum_finish(struct symbol *func) {}
#endif /* !BUILD_KLP */
#endif /* _OBJTOOL_CHECKSUM_H */

25
tools/objtool/include/objtool/checksum_types.h Normal file
View File

@@ -0,0 +1,25 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _OBJTOOL_CHECKSUM_TYPES_H
#define _OBJTOOL_CHECKSUM_TYPES_H
struct sym_checksum {
u64 addr;
u64 checksum;
};
#ifdef BUILD_KLP
#include <xxhash.h>
struct checksum {
XXH3_state_t *state;
XXH64_hash_t checksum;
};
#else /* !BUILD_KLP */
struct checksum {};
#endif /* !BUILD_KLP */
#endif /* _OBJTOOL_CHECKSUM_TYPES_H */

81
tools/objtool/include/objtool/disas.h Normal file
View File

@@ -0,0 +1,81 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright (c) 2025, Oracle and/or its affiliates.
*/
#ifndef _DISAS_H
#define _DISAS_H
struct alternative;
struct disas_context;
struct disassemble_info;
#ifdef DISAS
struct disas_context *disas_context_create(struct objtool_file *file);
void disas_context_destroy(struct disas_context *dctx);
void disas_warned_funcs(struct disas_context *dctx);
void disas_funcs(struct disas_context *dctx);
int disas_info_init(struct disassemble_info *dinfo,
int arch, int mach32, int mach64,
const char *options);
size_t disas_insn(struct disas_context *dctx, struct instruction *insn);
char *disas_result(struct disas_context *dctx);
void disas_print_info(FILE *stream, struct instruction *insn, int depth,
const char *format, ...);
void disas_print_insn(FILE *stream, struct disas_context *dctx,
struct instruction *insn, int depth,
const char *format, ...);
char *disas_alt_name(struct alternative *alt);
const char *disas_alt_type_name(struct instruction *insn);
#else /* DISAS */
#include <objtool/warn.h>
static inline struct disas_context *disas_context_create(struct objtool_file *file)
{
WARN("Rebuild with libopcodes for disassembly support");
return NULL;
}
static inline void disas_context_destroy(struct disas_context *dctx) {}
static inline void disas_warned_funcs(struct disas_context *dctx) {}
static inline void disas_funcs(struct disas_context *dctx) {}
static inline int disas_info_init(struct disassemble_info *dinfo,
int arch, int mach32, int mach64,
const char *options)
{
return -1;
}
static inline size_t disas_insn(struct disas_context *dctx,
struct instruction *insn)
{
return -1;
}
static inline char *disas_result(struct disas_context *dctx)
{
return NULL;
}
static inline void disas_print_info(FILE *stream, struct instruction *insn,
int depth, const char *format, ...) {}
static inline void disas_print_insn(FILE *stream, struct disas_context *dctx,
struct instruction *insn, int depth,
const char *format, ...) {}
static inline char *disas_alt_name(struct alternative *alt)
{
return NULL;
}
static inline const char *disas_alt_type_name(struct instruction *insn)
{
return NULL;
}
#endif /* DISAS */
#endif /* _DISAS_H */

198
tools/objtool/include/objtool/elf.h
View File

@@ -8,12 +8,21 @@
#include <stdio.h>
#include <gelf.h>
#include <linux/string.h>
#include <linux/list.h>
#include <linux/hashtable.h>
#include <linux/rbtree.h>
#include <linux/jhash.h>
#include <objtool/endianness.h>
#include <objtool/checksum_types.h>
#include <arch/elf.h>
#define SEC_NAME_LEN 1024
#define SYM_NAME_LEN 512
#define bswap_if_needed(elf, val) __bswap_if_needed(&elf->ehdr, val)
#ifdef LIBELF_USE_DEPRECATED
# define elf_getshdrnum elf_getshnum
# define elf_getshdrstrndx elf_getshstrndx
@@ -40,24 +49,27 @@ struct section {
struct section *base, *rsec;
struct symbol *sym;
Elf_Data *data;
char *name;
const char *name;
int idx;
bool _changed, text, rodata, noinstr, init, truncate;
struct reloc *relocs;
unsigned long nr_alloc_relocs;
struct section *twin;
};
struct symbol {
struct list_head list;
struct list_head global_list;
struct rb_node node;
struct elf_hash_node hash;
struct elf_hash_node name_hash;
GElf_Sym sym;
struct section *sec;
char *name;
const char *name, *demangled_name;
unsigned int idx, len;
unsigned long offset;
unsigned long __subtree_last;
struct symbol *pfunc, *cfunc, *alias;
struct symbol *pfunc, *cfunc, *alias, *file;
unsigned char bind, type;
u8 uaccess_safe : 1;
u8 static_call_tramp : 1;
@@ -71,9 +83,17 @@ struct symbol {
u8 frame_pointer : 1;
u8 ignore : 1;
u8 nocfi : 1;
u8 cold : 1;
u8 prefix : 1;
u8 debug_checksum : 1;
u8 changed : 1;
u8 included : 1;
u8 klp : 1;
struct list_head pv_target;
struct reloc *relocs;
struct section *group_sec;
struct checksum csum;
struct symbol *twin, *clone;
};
struct reloc {
@@ -88,9 +108,10 @@ struct elf {
GElf_Ehdr ehdr;
int fd;
bool changed;
char *name;
const char *name, *tmp_name;
unsigned int num_files;
struct list_head sections;
struct list_head symbols;
unsigned long num_relocs;
int symbol_bits;
@@ -110,14 +131,37 @@ struct elf {
};
struct elf *elf_open_read(const char *name, int flags);
struct elf *elf_create_file(GElf_Ehdr *ehdr, const char *name);
struct section *elf_create_section(struct elf *elf, const char *name,
size_t entsize, unsigned int nr);
size_t size, size_t entsize,
unsigned int type, unsigned int align,
unsigned int flags);
struct section *elf_create_section_pair(struct elf *elf, const char *name,
size_t entsize, unsigned int nr,
unsigned int reloc_nr);
struct symbol *elf_create_prefix_symbol(struct elf *elf, struct symbol *orig, long size);
struct section *elf_create_rela_section(struct elf *elf, struct section *sec,
unsigned int reloc_nr);
struct symbol *elf_create_symbol(struct elf *elf, const char *name,
struct section *sec, unsigned int bind,
unsigned int type, unsigned long offset,
size_t size);
struct symbol *elf_create_section_symbol(struct elf *elf, struct section *sec);
void *elf_add_data(struct elf *elf, struct section *sec, const void *data,
size_t size);
unsigned int elf_add_string(struct elf *elf, struct section *strtab, const char *str);
struct reloc *elf_create_reloc(struct elf *elf, struct section *sec,
unsigned long offset, struct symbol *sym,
s64 addend, unsigned int type);
struct reloc *elf_init_reloc(struct elf *elf, struct section *rsec,
unsigned int reloc_idx, unsigned long offset,
struct symbol *sym, s64 addend, unsigned int type);
struct reloc *elf_init_reloc_text_sym(struct elf *elf, struct section *sec,
unsigned long offset,
@@ -131,16 +175,17 @@ struct reloc *elf_init_reloc_data_sym(struct elf *elf, struct section *sec,
struct symbol *sym,
s64 addend);
int elf_write_insn(struct elf *elf, struct section *sec,
unsigned long offset, unsigned int len,
const char *insn);
int elf_write_insn(struct elf *elf, struct section *sec, unsigned long offset,
unsigned int len, const char *insn);
int elf_write(struct elf *elf);
void elf_close(struct elf *elf);
int elf_close(struct elf *elf);
struct section *find_section_by_name(const struct elf *elf, const char *name);
struct symbol *find_func_by_offset(struct section *sec, unsigned long offset);
struct symbol *find_symbol_by_offset(struct section *sec, unsigned long offset);
struct symbol *find_symbol_by_name(const struct elf *elf, const char *name);
struct symbol *find_global_symbol_by_name(const struct elf *elf, const char *name);
struct symbol *find_symbol_containing(const struct section *sec, unsigned long offset);
int find_symbol_hole_containing(const struct section *sec, unsigned long offset);
struct reloc *find_reloc_by_dest(const struct elf *elf, struct section *sec, unsigned long offset);
@@ -178,11 +223,76 @@ static inline unsigned int elf_text_rela_type(struct elf *elf)
return elf_addr_size(elf) == 4 ? R_TEXT32 : R_TEXT64;
}
static inline bool is_undef_sym(struct symbol *sym)
{
return !sym->sec->idx;
}
static inline bool is_null_sym(struct symbol *sym)
{
return !sym->idx;
}
static inline bool is_sec_sym(struct symbol *sym)
{
return sym->type == STT_SECTION;
}
static inline bool is_object_sym(struct symbol *sym)
{
return sym->type == STT_OBJECT;
}
static inline bool is_func_sym(struct symbol *sym)
{
return sym->type == STT_FUNC;
}
static inline bool is_file_sym(struct symbol *sym)
{
return sym->type == STT_FILE;
}
static inline bool is_notype_sym(struct symbol *sym)
{
return sym->type == STT_NOTYPE;
}
static inline bool is_global_sym(struct symbol *sym)
{
return sym->bind == STB_GLOBAL;
}
static inline bool is_weak_sym(struct symbol *sym)
{
return sym->bind == STB_WEAK;
}
static inline bool is_local_sym(struct symbol *sym)
{
return sym->bind == STB_LOCAL;
}
static inline bool is_prefix_func(struct symbol *sym)
{
return sym->prefix;
}
static inline bool is_reloc_sec(struct section *sec)
{
return sec->sh.sh_type == SHT_RELA || sec->sh.sh_type == SHT_REL;
}
static inline bool is_string_sec(struct section *sec)
{
return sec->sh.sh_flags & SHF_STRINGS;
}
static inline bool is_text_sec(struct section *sec)
{
return sec->sh.sh_flags & SHF_EXECINSTR;
}
static inline bool sec_changed(struct section *sec)
{
return sec->_changed;
@@ -223,6 +333,11 @@ static inline bool is_32bit_reloc(struct reloc *reloc)
return reloc->sec->sh.sh_entsize < 16;
}
static inline unsigned long sec_size(struct section *sec)
{
return sec->sh.sh_size;
}
#define __get_reloc_field(reloc, field) \
({ \
is_32bit_reloc(reloc) ? \
@@ -300,6 +415,15 @@ static inline void set_reloc_type(struct elf *elf, struct reloc *reloc, unsigned
mark_sec_changed(elf, reloc->sec, true);
}
static inline unsigned int annotype(struct elf *elf, struct section *sec,
struct reloc *reloc)
{
unsigned int type;
type = *(u32 *)(sec->data->d_buf + (reloc_idx(reloc) * 8) + 4);
return bswap_if_needed(elf, type);
}
#define RELOC_JUMP_TABLE_BIT 1UL
/* Does reloc mark the beginning of a jump table? */
@@ -325,28 +449,54 @@ static inline void set_sym_next_reloc(struct reloc *reloc, struct reloc *next)
reloc->_sym_next_reloc = (unsigned long)next | bit;
}
#define for_each_sec(file, sec) \
list_for_each_entry(sec, &file->elf->sections, list)
#define for_each_sec(elf, sec) \
list_for_each_entry(sec, &elf->sections, list)
#define sec_for_each_sym(sec, sym) \
list_for_each_entry(sym, &sec->symbol_list, list)
#define for_each_sym(file, sym) \
for (struct section *__sec, *__fake = (struct section *)1; \
__fake; __fake = NULL) \
for_each_sec(file, __sec) \
sec_for_each_sym(__sec, sym)
#define sec_prev_sym(sym) \
sym->sec && sym->list.prev != &sym->sec->symbol_list ? \
list_prev_entry(sym, list) : NULL
#define for_each_sym(elf, sym) \
list_for_each_entry(sym, &elf->symbols, global_list)
#define for_each_sym_continue(elf, sym) \
list_for_each_entry_continue(sym, &elf->symbols, global_list)
#define rsec_next_reloc(rsec, reloc) \
reloc_idx(reloc) < sec_num_entries(rsec) - 1 ? reloc + 1 : NULL
#define for_each_reloc(rsec, reloc) \
for (int __i = 0, __fake = 1; __fake; __fake = 0) \
for (reloc = rsec->relocs; \
__i < sec_num_entries(rsec); \
__i++, reloc++)
for (reloc = rsec->relocs; reloc; reloc = rsec_next_reloc(rsec, reloc))
#define for_each_reloc_from(rsec, reloc) \
for (int __i = reloc_idx(reloc); \
__i < sec_num_entries(rsec); \
__i++, reloc++)
for (; reloc; reloc = rsec_next_reloc(rsec, reloc))
#define for_each_reloc_continue(rsec, reloc) \
for (reloc = rsec_next_reloc(rsec, reloc); reloc; \
reloc = rsec_next_reloc(rsec, reloc))
#define sym_for_each_reloc(elf, sym, reloc) \
for (reloc = find_reloc_by_dest_range(elf, sym->sec, \
sym->offset, sym->len); \
reloc && reloc_offset(reloc) < sym->offset + sym->len; \
reloc = rsec_next_reloc(sym->sec->rsec, reloc))
static inline struct symbol *get_func_prefix(struct symbol *func)
{
struct symbol *prev;
if (!is_func_sym(func))
return NULL;
prev = sec_prev_sym(func);
if (prev && is_prefix_func(prev))
return prev;
return NULL;
}
#define OFFSET_STRIDE_BITS 4
#define OFFSET_STRIDE (1UL << OFFSET_STRIDE_BITS)

9
tools/objtool/include/objtool/endianness.h
View File

@@ -4,7 +4,6 @@
#include <linux/kernel.h>
#include <endian.h>
#include <objtool/elf.h>
/*
* Does a byte swap if target file endianness doesn't match the host, i.e. cross
@@ -12,16 +11,16 @@
* To be used for multi-byte values conversion, which are read from / about
* to be written to a target native endianness ELF file.
*/
static inline bool need_bswap(struct elf *elf)
static inline bool need_bswap(GElf_Ehdr *ehdr)
{
return (__BYTE_ORDER == __LITTLE_ENDIAN) ^
(elf->ehdr.e_ident[EI_DATA] == ELFDATA2LSB);
(ehdr->e_ident[EI_DATA] == ELFDATA2LSB);
}
#define bswap_if_needed(elf, val) \
#define __bswap_if_needed(ehdr, val) \
({ \
__typeof__(val) __ret; \
bool __need_bswap = need_bswap(elf); \
bool __need_bswap = need_bswap(ehdr); \
switch (sizeof(val)) { \
case 8: \
__ret = __need_bswap ? bswap_64(val) : (val); break; \

35
tools/objtool/include/objtool/klp.h Normal file
View File

@@ -0,0 +1,35 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
#ifndef _OBJTOOL_KLP_H
#define _OBJTOOL_KLP_H
#define SHF_RELA_LIVEPATCH 0x00100000
#define SHN_LIVEPATCH 0xff20
/*
* __klp_objects and __klp_funcs are created by klp diff and used by the patch
* module init code to build the klp_patch, klp_object and klp_func structs
* needed by the livepatch API.
*/
#define KLP_OBJECTS_SEC "__klp_objects"
#define KLP_FUNCS_SEC "__klp_funcs"
/*
* __klp_relocs is an intermediate section which are created by klp diff and
* converted into KLP symbols/relas by "objtool klp post-link". This is needed
* to work around the linker, which doesn't preserve SHN_LIVEPATCH or
* SHF_RELA_LIVEPATCH, nor does it support having two RELA sections for a
* single PROGBITS section.
*/
#define KLP_RELOCS_SEC "__klp_relocs"
#define KLP_STRINGS_SEC ".rodata.klp.str1.1"
struct klp_reloc {
void *offset;
void *sym;
u32 type;
};
int cmd_klp_diff(int argc, const char **argv);
int cmd_klp_post_link(int argc, const char **argv);
#endif /* _OBJTOOL_KLP_H */

4
tools/objtool/include/objtool/objtool.h
View File

@@ -28,7 +28,7 @@ struct objtool_file {
struct list_head mcount_loc_list;
struct list_head endbr_list;
struct list_head call_list;
bool ignore_unreachables, hints, rodata;
bool ignore_unreachables, hints, rodata, klp;
unsigned int nr_endbr;
unsigned int nr_endbr_int;
@@ -39,6 +39,8 @@ struct objtool_file {
struct pv_state *pv_ops;
};
char *top_level_dir(const char *file);
struct objtool_file *objtool_open_read(const char *_objname);
int objtool_pv_add(struct objtool_file *file, int idx, struct symbol *func);

4
tools/objtool/include/objtool/special.h
View File

@@ -25,7 +25,7 @@ struct special_alt {
struct section *new_sec;
unsigned long new_off;
unsigned int orig_len, new_len; /* group only */
unsigned int orig_len, new_len, feature; /* group only */
};
int special_get_alts(struct elf *elf, struct list_head *alts);
@@ -38,4 +38,6 @@ bool arch_support_alt_relocation(struct special_alt *special_alt,
struct reloc *arch_find_switch_table(struct objtool_file *file,
struct instruction *insn,
unsigned long *table_size);
const char *arch_cpu_feature_name(int feature_number);
#endif /* _SPECIAL_H */

141
tools/objtool/include/objtool/trace.h Normal file
View File

@@ -0,0 +1,141 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright (c) 2025, Oracle and/or its affiliates.
*/
#ifndef _TRACE_H
#define _TRACE_H
#include <objtool/check.h>
#include <objtool/disas.h>
#ifdef DISAS
extern bool trace;
extern int trace_depth;
#define TRACE(fmt, ...) \
({ if (trace) \
fprintf(stderr, fmt, ##__VA_ARGS__); \
})
/*
* Print the instruction address and a message. The instruction
* itself is not printed.
*/
#define TRACE_ADDR(insn, fmt, ...) \
({ \
if (trace) { \
disas_print_info(stderr, insn, trace_depth - 1, \
fmt "\n", ##__VA_ARGS__); \
} \
})
/*
* Print the instruction address, the instruction and a message.
*/
#define TRACE_INSN(insn, fmt, ...) \
({ \
if (trace) { \
disas_print_insn(stderr, objtool_disas_ctx, \
insn, trace_depth - 1, \
fmt, ##__VA_ARGS__); \
fprintf(stderr, "\n"); \
insn->trace = 1; \
} \
})
#define TRACE_INSN_STATE(insn, sprev, snext) \
({ \
if (trace) \
trace_insn_state(insn, sprev, snext); \
})
#define TRACE_ALT_FMT(pfx, fmt) pfx "<%s.%lx> " fmt
#define TRACE_ALT_ARG(insn) disas_alt_type_name(insn), (insn)->offset
#define TRACE_ALT(insn, fmt, ...) \
TRACE_INSN(insn, TRACE_ALT_FMT("", fmt), \
TRACE_ALT_ARG(insn), ##__VA_ARGS__)
#define TRACE_ALT_INFO(insn, pfx, fmt, ...) \
TRACE_ADDR(insn, TRACE_ALT_FMT(pfx, fmt), \
TRACE_ALT_ARG(insn), ##__VA_ARGS__)
#define TRACE_ALT_INFO_NOADDR(insn, pfx, fmt, ...) \
TRACE_ADDR(NULL, TRACE_ALT_FMT(pfx, fmt), \
TRACE_ALT_ARG(insn), ##__VA_ARGS__)
#define TRACE_ALT_BEGIN(insn, alt, alt_name) \
({ \
if (trace) { \
alt_name = disas_alt_name(alt); \
trace_alt_begin(insn, alt, alt_name); \
} \
})
#define TRACE_ALT_END(insn, alt, alt_name) \
({ \
if (trace) { \
trace_alt_end(insn, alt, alt_name); \
free(alt_name); \
} \
})
static inline void trace_enable(void)
{
trace = true;
trace_depth = 0;
}
static inline void trace_disable(void)
{
trace = false;
}
static inline void trace_depth_inc(void)
{
if (trace)
trace_depth++;
}
static inline void trace_depth_dec(void)
{
if (trace)
trace_depth--;
}
void trace_insn_state(struct instruction *insn, struct insn_state *sprev,
struct insn_state *snext);
void trace_alt_begin(struct instruction *orig_insn, struct alternative *alt,
char *alt_name);
void trace_alt_end(struct instruction *orig_insn, struct alternative *alt,
char *alt_name);
#else /* DISAS */
#define TRACE(fmt, ...) ({})
#define TRACE_ADDR(insn, fmt, ...) ({})
#define TRACE_INSN(insn, fmt, ...) ({})
#define TRACE_INSN_STATE(insn, sprev, snext) ({})
#define TRACE_ALT(insn, fmt, ...) ({})
#define TRACE_ALT_INFO(insn, fmt, ...) ({})
#define TRACE_ALT_INFO_NOADDR(insn, fmt, ...) ({})
#define TRACE_ALT_BEGIN(insn, alt, alt_name) ({})
#define TRACE_ALT_END(insn, alt, alt_name) ({})
static inline void trace_enable(void) {}
static inline void trace_disable(void) {}
static inline void trace_depth_inc(void) {}
static inline void trace_depth_dec(void) {}
static inline void trace_alt_begin(struct instruction *orig_insn,
struct alternative *alt,
char *alt_name) {};
static inline void trace_alt_end(struct instruction *orig_insn,
struct alternative *alt,
char *alt_name) {};
#endif
#endif /* _TRACE_H */

19
tools/objtool/include/objtool/util.h Normal file
View File

@@ -0,0 +1,19 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
#ifndef _UTIL_H
#define _UTIL_H
#include <objtool/warn.h>
#define snprintf_check(str, size, format, args...) \
({ \
int __ret = snprintf(str, size, format, args); \
if (__ret < 0) \
ERROR_GLIBC("snprintf"); \
else if (__ret >= size) \
ERROR("snprintf() failed for '" format "'", args); \
else \
__ret = 0; \
__ret; \
})
#endif /* _UTIL_H */

66
tools/objtool/include/objtool/warn.h
View File

@@ -77,9 +77,11 @@ static inline char *offstr(struct section *sec, unsigned long offset)
#define WARN_INSN(insn, format, ...) \
({ \
struct instruction *_insn = (insn); \
if (!_insn->sym || !_insn->sym->warned) \
if (!_insn->sym || !_insn->sym->warned) { \
WARN_FUNC(_insn->sec, _insn->offset, format, \
##__VA_ARGS__); \
BT_INSN(_insn, ""); \
} \
if (_insn->sym) \
_insn->sym->warned = 1; \
})
@@ -87,10 +89,15 @@ static inline char *offstr(struct section *sec, unsigned long offset)
#define BT_INSN(insn, format, ...) \
({ \
if (opts.verbose || opts.backtrace) { \
struct instruction *_insn = (insn); \
char *_str = offstr(_insn->sec, _insn->offset); \
WARN(" %s: " format, _str, ##__VA_ARGS__); \
free(_str); \
struct instruction *__insn = (insn); \
char *_str = offstr(__insn->sec, __insn->offset); \
const char *_istr = objtool_disas_insn(__insn); \
int _len; \
_len = snprintf(NULL, 0, " %s: " format, _str, ##__VA_ARGS__); \
_len = (_len < 50) ? 50 - _len : 0; \
WARN(" %s: " format " %*s%s", _str, ##__VA_ARGS__, _len, "", _istr); \
free(_str); \
__insn->trace = 1; \
} \
})
@@ -102,4 +109,53 @@ static inline char *offstr(struct section *sec, unsigned long offset)
#define ERROR_FUNC(sec, offset, format, ...) __WARN_FUNC(ERROR_STR, sec, offset, format, ##__VA_ARGS__)
#define ERROR_INSN(insn, format, ...) WARN_FUNC(insn->sec, insn->offset, format, ##__VA_ARGS__)
extern bool debug;
extern int indent;
static inline void unindent(int *unused) { indent--; }
/*
* Clang prior to 17 is being silly and considers many __cleanup() variables
* as unused (because they are, their sole purpose is to go out of scope).
*
* https://github.com/llvm/llvm-project/commit/877210faa447f4cc7db87812f8ed80e398fedd61
*/
#undef __cleanup
#define __cleanup(func) __maybe_unused __attribute__((__cleanup__(func)))
#define __dbg(format, ...) \
fprintf(stderr, \
"DEBUG: %s%s" format "\n", \
objname ?: "", \
objname ? ": " : "", \
##__VA_ARGS__)
#define dbg(args...) \
({ \
if (unlikely(debug)) \
__dbg(args); \
})
#define __dbg_indent(format, ...) \
({ \
if (unlikely(debug)) \
__dbg("%*s" format, indent * 8, "", ##__VA_ARGS__); \
})
#define dbg_indent(args...) \
int __cleanup(unindent) __dummy_##__COUNTER__; \
__dbg_indent(args); \
indent++
#define dbg_checksum(func, insn, checksum) \
({ \
if (unlikely(insn->sym && insn->sym->pfunc && \
insn->sym->pfunc->debug_checksum)) { \
char *insn_off = offstr(insn->sec, insn->offset); \
__dbg("checksum: %s %s %016lx", \
func->name, insn_off, checksum); \
free(insn_off); \
} \
})
#endif /* _WARN_H */

1723
tools/objtool/klp-diff.c Normal file
View File

File diff suppressed because it is too large Load Diff

168
tools/objtool/klp-post-link.c Normal file
View File

@@ -0,0 +1,168 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Read the intermediate KLP reloc/symbol representations created by klp diff
* and convert them to the proper format required by livepatch. This needs to
* run last to avoid linker wreckage. Linkers don't tend to handle the "two
* rela sections for a single base section" case very well, nor do they like
* SHN_LIVEPATCH.
*
* This is the final tool in the livepatch module generation pipeline:
*
* kernel builds -> objtool klp diff -> module link -> objtool klp post-link
*/
#include <fcntl.h>
#include <gelf.h>
#include <objtool/objtool.h>
#include <objtool/warn.h>
#include <objtool/klp.h>
#include <objtool/util.h>
#include <linux/livepatch_external.h>
static int fix_klp_relocs(struct elf *elf)
{
struct section *symtab, *klp_relocs;
klp_relocs = find_section_by_name(elf, KLP_RELOCS_SEC);
if (!klp_relocs)
return 0;
symtab = find_section_by_name(elf, ".symtab");
if (!symtab) {
ERROR("missing .symtab");
return -1;
}
for (int i = 0; i < sec_size(klp_relocs) / sizeof(struct klp_reloc); i++) {
struct klp_reloc *klp_reloc;
unsigned long klp_reloc_off;
struct section *sec, *tmp, *klp_rsec;
unsigned long offset;
struct reloc *reloc;
char sym_modname[64];
char rsec_name[SEC_NAME_LEN];
u64 addend;
struct symbol *sym, *klp_sym;
klp_reloc_off = i * sizeof(*klp_reloc);
klp_reloc = klp_relocs->data->d_buf + klp_reloc_off;
/*
* Read __klp_relocs[i]:
*/
/* klp_reloc.sec_offset */
reloc = find_reloc_by_dest(elf, klp_relocs,
klp_reloc_off + offsetof(struct klp_reloc, offset));
if (!reloc) {
ERROR("malformed " KLP_RELOCS_SEC " section");
return -1;
}
sec = reloc->sym->sec;
offset = reloc_addend(reloc);
/* klp_reloc.sym */
reloc = find_reloc_by_dest(elf, klp_relocs,
klp_reloc_off + offsetof(struct klp_reloc, sym));
if (!reloc) {
ERROR("malformed " KLP_RELOCS_SEC " section");
return -1;
}
klp_sym = reloc->sym;
addend = reloc_addend(reloc);
/* symbol format: .klp.sym.modname.sym_name,sympos */
if (sscanf(klp_sym->name + strlen(KLP_SYM_PREFIX), "%55[^.]", sym_modname) != 1)
ERROR("can't find modname in klp symbol '%s'", klp_sym->name);
/*
* Create the KLP rela:
*/
/* section format: .klp.rela.sec_objname.section_name */
if (snprintf_check(rsec_name, SEC_NAME_LEN,
KLP_RELOC_SEC_PREFIX "%s.%s",
sym_modname, sec->name))
return -1;
klp_rsec = find_section_by_name(elf, rsec_name);
if (!klp_rsec) {
klp_rsec = elf_create_section(elf, rsec_name, 0,
elf_rela_size(elf),
SHT_RELA, elf_addr_size(elf),
SHF_ALLOC | SHF_INFO_LINK | SHF_RELA_LIVEPATCH);
if (!klp_rsec)
return -1;
klp_rsec->sh.sh_link = symtab->idx;
klp_rsec->sh.sh_info = sec->idx;
klp_rsec->base = sec;
}
tmp = sec->rsec;
sec->rsec = klp_rsec;
if (!elf_create_reloc(elf, sec, offset, klp_sym, addend, klp_reloc->type))
return -1;
sec->rsec = tmp;
/*
* Fix up the corresponding KLP symbol:
*/
klp_sym->sym.st_shndx = SHN_LIVEPATCH;
if (!gelf_update_sym(symtab->data, klp_sym->idx, &klp_sym->sym)) {
ERROR_ELF("gelf_update_sym");
return -1;
}
/*
* Disable the original non-KLP reloc by converting it to R_*_NONE:
*/
reloc = find_reloc_by_dest(elf, sec, offset);
sym = reloc->sym;
sym->sym.st_shndx = SHN_LIVEPATCH;
set_reloc_type(elf, reloc, 0);
if (!gelf_update_sym(symtab->data, sym->idx, &sym->sym)) {
ERROR_ELF("gelf_update_sym");
return -1;
}
}
return 0;
}
/*
* This runs on the livepatch module after all other linking has been done. It
* converts the intermediate __klp_relocs section into proper KLP relocs to be
* processed by livepatch. This needs to run last to avoid linker wreckage.
* Linkers don't tend to handle the "two rela sections for a single base
* section" case very well, nor do they appreciate SHN_LIVEPATCH.
*/
int cmd_klp_post_link(int argc, const char **argv)
{
struct elf *elf;
argc--;
argv++;
if (argc != 1) {
fprintf(stderr, "%d\n", argc);
fprintf(stderr, "usage: objtool link <file.ko>\n");
return -1;
}
elf = elf_open_read(argv[0], O_RDWR);
if (!elf)
return -1;
if (fix_klp_relocs(elf))
return -1;
if (elf_write(elf))
return -1;
return elf_close(elf);
}

1
tools/objtool/noreturns.h
View File

@@ -36,6 +36,7 @@ NORETURN(machine_real_restart)
NORETURN(make_task_dead)
NORETURN(mpt_halt_firmware)
NORETURN(mwait_play_dead)
NORETURN(native_play_dead)
NORETURN(nmi_panic_self_stop)
NORETURN(panic)
NORETURN(vpanic)

42
tools/objtool/objtool.c
View File

@@ -16,7 +16,8 @@
#include <objtool/objtool.h>
#include <objtool/warn.h>
bool help;
bool debug;
int indent;
static struct objtool_file file;
@@ -71,6 +72,39 @@ int objtool_pv_add(struct objtool_file *f, int idx, struct symbol *func)
return 0;
}
char *top_level_dir(const char *file)
{
ssize_t len, self_len, file_len;
char self[PATH_MAX], *str;
int i;
len = readlink("/proc/self/exe", self, sizeof(self) - 1);
if (len <= 0)
return NULL;
self[len] = '\0';
for (i = 0; i < 3; i++) {
char *s = strrchr(self, '/');
if (!s)
return NULL;
*s = '\0';
}
self_len = strlen(self);
file_len = strlen(file);
str = malloc(self_len + file_len + 2);
if (!str)
return NULL;
memcpy(str, self, self_len);
str[self_len] = '/';
strcpy(str + self_len + 1, file);
return str;
}
int main(int argc, const char **argv)
{
static const char *UNUSED = "OBJTOOL_NOT_IMPLEMENTED";
@@ -79,5 +113,11 @@ int main(int argc, const char **argv)
exec_cmd_init("objtool", UNUSED, UNUSED, UNUSED);
pager_init(UNUSED);
if (argc > 1 && !strcmp(argv[1], "klp")) {
argc--;
argv++;
return cmd_klp(argc, argv);
}
return objtool_run(argc, argv);
}

1
tools/objtool/orc_dump.c
View File

@@ -8,7 +8,6 @@
#include <objtool/objtool.h>
#include <objtool/orc.h>
#include <objtool/warn.h>
#include <objtool/endianness.h>
int orc_dump(const char *filename)
{

9
tools/objtool/orc_gen.c
View File

@@ -12,7 +12,6 @@
#include <objtool/check.h>
#include <objtool/orc.h>
#include <objtool/warn.h>
#include <objtool/endianness.h>
struct orc_list_entry {
struct list_head list;
@@ -57,7 +56,7 @@ int orc_create(struct objtool_file *file)
/* Build a deduplicated list of ORC entries: */
INIT_LIST_HEAD(&orc_list);
for_each_sec(file, sec) {
for_each_sec(file->elf, sec) {
struct orc_entry orc, prev_orc = {0};
struct instruction *insn;
bool empty = true;
@@ -127,7 +126,11 @@ int orc_create(struct objtool_file *file)
return -1;
}
orc_sec = elf_create_section(file->elf, ".orc_unwind",
sizeof(struct orc_entry), nr);
nr * sizeof(struct orc_entry),
sizeof(struct orc_entry),
SHT_PROGBITS,
1,
SHF_ALLOC);
if (!orc_sec)
return -1;

16
tools/objtool/special.c
View File

@@ -15,7 +15,6 @@
#include <objtool/builtin.h>
#include <objtool/special.h>
#include <objtool/warn.h>
#include <objtool/endianness.h>
struct special_entry {
const char *sec;
@@ -82,6 +81,8 @@ static int get_alt_entry(struct elf *elf, const struct special_entry *entry,
entry->orig_len);
alt->new_len = *(unsigned char *)(sec->data->d_buf + offset +
entry->new_len);
alt->feature = *(unsigned int *)(sec->data->d_buf + offset +
entry->feature);
}
orig_reloc = find_reloc_by_dest(elf, sec, offset + entry->orig);
@@ -133,7 +134,7 @@ int special_get_alts(struct elf *elf, struct list_head *alts)
struct section *sec;
unsigned int nr_entries;
struct special_alt *alt;
int idx, ret;
int idx;
INIT_LIST_HEAD(alts);
@@ -142,12 +143,12 @@ int special_get_alts(struct elf *elf, struct list_head *alts)
if (!sec)
continue;
if (sec->sh.sh_size % entry->size != 0) {
if (sec_size(sec) % entry->size != 0) {
ERROR("%s size not a multiple of %d", sec->name, entry->size);
return -1;
}
nr_entries = sec->sh.sh_size / entry->size;
nr_entries = sec_size(sec) / entry->size;
for (idx = 0; idx < nr_entries; idx++) {
alt = malloc(sizeof(*alt));
@@ -157,11 +158,8 @@ int special_get_alts(struct elf *elf, struct list_head *alts)
}
memset(alt, 0, sizeof(*alt));
ret = get_alt_entry(elf, entry, sec, idx, alt);
if (ret > 0)
continue;
if (ret < 0)
return ret;
if (get_alt_entry(elf, entry, sec, idx, alt))
return -1;
list_add_tail(&alt->list, alts);
}

2
tools/objtool/sync-check.sh
View File

@@ -16,6 +16,8 @@ arch/x86/include/asm/orc_types.h
arch/x86/include/asm/emulate_prefix.h
arch/x86/lib/x86-opcode-map.txt
arch/x86/tools/gen-insn-attr-x86.awk
include/linux/interval_tree_generic.h
include/linux/livepatch_external.h
include/linux/static_call_types.h
"

203
tools/objtool/trace.c Normal file
View File

@@ -0,0 +1,203 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (c) 2025, Oracle and/or its affiliates.
*/
#include <objtool/trace.h>
bool trace;
int trace_depth;
/*
* Macros to trace CFI state attributes changes.
*/
#define TRACE_CFI_ATTR(attr, prev, next, fmt, ...) \
({ \
if ((prev)->attr != (next)->attr) \
TRACE("%s=" fmt " ", #attr, __VA_ARGS__); \
})
#define TRACE_CFI_ATTR_BOOL(attr, prev, next) \
TRACE_CFI_ATTR(attr, prev, next, \
"%s", (next)->attr ? "true" : "false")
#define TRACE_CFI_ATTR_NUM(attr, prev, next, fmt) \
TRACE_CFI_ATTR(attr, prev, next, fmt, (next)->attr)
#define CFI_REG_NAME_MAXLEN 16
/*
* Return the name of a register. Note that the same static buffer
* is returned if the name is dynamically generated.
*/
static const char *cfi_reg_name(unsigned int reg)
{
static char rname_buffer[CFI_REG_NAME_MAXLEN];
const char *rname;
switch (reg) {
case CFI_UNDEFINED:
return "<undefined>";
case CFI_CFA:
return "cfa";
case CFI_SP_INDIRECT:
return "(sp)";
case CFI_BP_INDIRECT:
return "(bp)";
}
if (reg < CFI_NUM_REGS) {
rname = arch_reg_name[reg];
if (rname)
return rname;
}
if (snprintf(rname_buffer, CFI_REG_NAME_MAXLEN, "r%d", reg) == -1)
return "<error>";
return (const char *)rname_buffer;
}
/*
* Functions and macros to trace CFI registers changes.
*/
static void trace_cfi_reg(const char *prefix, int reg, const char *fmt,
int base_prev, int offset_prev,
int base_next, int offset_next)
{
char *rname;
if (base_prev == base_next && offset_prev == offset_next)
return;
if (prefix)
TRACE("%s:", prefix);
if (base_next == CFI_UNDEFINED) {
TRACE("%1$s=<undef> ", cfi_reg_name(reg));
} else {
rname = strdup(cfi_reg_name(reg));
TRACE(fmt, rname, cfi_reg_name(base_next), offset_next);
free(rname);
}
}
static void trace_cfi_reg_val(const char *prefix, int reg,
int base_prev, int offset_prev,
int base_next, int offset_next)
{
trace_cfi_reg(prefix, reg, "%1$s=%2$s%3$+d ",
base_prev, offset_prev, base_next, offset_next);
}
static void trace_cfi_reg_ref(const char *prefix, int reg,
int base_prev, int offset_prev,
int base_next, int offset_next)
{
trace_cfi_reg(prefix, reg, "%1$s=(%2$s%3$+d) ",
base_prev, offset_prev, base_next, offset_next);
}
#define TRACE_CFI_REG_VAL(reg, prev, next) \
trace_cfi_reg_val(NULL, reg, prev.base, prev.offset, \
next.base, next.offset)
#define TRACE_CFI_REG_REF(reg, prev, next) \
trace_cfi_reg_ref(NULL, reg, prev.base, prev.offset, \
next.base, next.offset)
void trace_insn_state(struct instruction *insn, struct insn_state *sprev,
struct insn_state *snext)
{
struct cfi_state *cprev, *cnext;
int i;
if (!memcmp(sprev, snext, sizeof(struct insn_state)))
return;
cprev = &sprev->cfi;
cnext = &snext->cfi;
disas_print_insn(stderr, objtool_disas_ctx, insn,
trace_depth - 1, "state: ");
/* print registers changes */
TRACE_CFI_REG_VAL(CFI_CFA, cprev->cfa, cnext->cfa);
for (i = 0; i < CFI_NUM_REGS; i++) {
TRACE_CFI_REG_VAL(i, cprev->vals[i], cnext->vals[i]);
TRACE_CFI_REG_REF(i, cprev->regs[i], cnext->regs[i]);
}
/* print attributes changes */
TRACE_CFI_ATTR_NUM(stack_size, cprev, cnext, "%d");
TRACE_CFI_ATTR_BOOL(drap, cprev, cnext);
if (cnext->drap) {
trace_cfi_reg_val("drap", cnext->drap_reg,
cprev->drap_reg, cprev->drap_offset,
cnext->drap_reg, cnext->drap_offset);
}
TRACE_CFI_ATTR_BOOL(bp_scratch, cprev, cnext);
TRACE_CFI_ATTR_NUM(instr, sprev, snext, "%d");
TRACE_CFI_ATTR_NUM(uaccess_stack, sprev, snext, "%u");
TRACE("\n");
insn->trace = 1;
}
void trace_alt_begin(struct instruction *orig_insn, struct alternative *alt,
char *alt_name)
{
struct instruction *alt_insn;
char suffix[2];
alt_insn = alt->insn;
if (alt->type == ALT_TYPE_EX_TABLE) {
/*
* When there is an exception table then the instruction
* at the original location is executed but it can cause
* an exception. In that case, the execution will be
* redirected to the alternative instruction.
*
* The instruction at the original location can have
* instruction alternatives, so we just print the location
* of the instruction that can cause the exception and
* not the instruction itself.
*/
TRACE_ALT_INFO_NOADDR(orig_insn, "/ ", "%s for instruction at 0x%lx <%s+0x%lx>",
alt_name,
orig_insn->offset, orig_insn->sym->name,
orig_insn->offset - orig_insn->sym->offset);
} else {
TRACE_ALT_INFO_NOADDR(orig_insn, "/ ", "%s", alt_name);
}
if (alt->type == ALT_TYPE_JUMP_TABLE) {
/*
* For a jump alternative, if the default instruction is
* a NOP then it is replaced with the jmp instruction,
* otherwise it is replaced with a NOP instruction.
*/
trace_depth++;
if (orig_insn->type == INSN_NOP) {
suffix[0] = (orig_insn->len == 5) ? 'q' : '\0';
TRACE_ADDR(orig_insn, "jmp%-3s %lx <%s+0x%lx>", suffix,
alt_insn->offset, alt_insn->sym->name,
alt_insn->offset - alt_insn->sym->offset);
} else {
TRACE_ADDR(orig_insn, "nop%d", orig_insn->len);
trace_depth--;
}
}
}
void trace_alt_end(struct instruction *orig_insn, struct alternative *alt,
char *alt_name)
{
if (alt->type == ALT_TYPE_JUMP_TABLE && orig_insn->type == INSN_NOP)
trace_depth--;
TRACE_ALT_INFO_NOADDR(orig_insn, "\\ ", "%s", alt_name);
}

7
tools/objtool/weak.c
View File

@@ -8,6 +8,8 @@
#include <stdbool.h>
#include <errno.h>
#include <objtool/objtool.h>
#include <objtool/arch.h>
#include <objtool/builtin.h>
#define UNSUPPORTED(name) \
({ \
@@ -24,3 +26,8 @@ int __weak orc_create(struct objtool_file *file)
{
UNSUPPORTED("ORC");
}
int __weak cmd_klp(int argc, const char **argv)
{
UNSUPPORTED("klp");
}

6
tools/perf/Makefile.perf
View File

@@ -234,12 +234,12 @@ endif
# The fixdep build - we force fixdep tool to be built as
# the first target in the separate make session not to be
# disturbed by any parallel make jobs. Once fixdep is done
# we issue the requested build with FIXDEP=1 variable.
# we issue the requested build with FIXDEP_BUILT=1 variable.
#
# The fixdep build is disabled for $(NON_CONFIG_TARGETS)
# targets, because it's not necessary.
ifdef FIXDEP
ifdef FIXDEP_BUILT
force_fixdep := 0
else
force_fixdep := $(config)
@@ -286,7 +286,7 @@ $(goals) all: sub-make
sub-make: fixdep
@./check-headers.sh
$(Q)$(MAKE) FIXDEP=1 -f Makefile.perf $(goals)
$(Q)$(MAKE) FIXDEP_BUILT=1 -f Makefile.perf $(goals)
else # force_fixdep