program: automatic verifier log sizing and ProgramInfo.RecursionMisses()

For the longest time users had to specify ProgramOptions.LogSize to get the full verifier log for large programs. The library now automatically figures out the correct buffer size to use and the field is deprecated. It will be removed in the next version.

ProgramInfo now exposes how often a program didn't execute due to recursion limits via RecursionMisses.

perf, ringbuf: better control over wakeups, Flush()

There is a trade off between how fast samples are processed and how much CPU is used. Reading samples as quickly as possible uses more CPU, reading in batches is cheaper.

It's now possible to configure a "maximum time before a wakeup" by using Reader.SetDeadline(). This now guarantees that any pending samples are read at the end of the deadline, even if the Watermark wasn't reached.

It's also possible to manually flush the buffer using Flush().

Bugfixes

• The various Copy() methods now to a full deep copy and all accept nil values.
• link: kprobe and uprobe links can now be pinned.

What's Changed

• perf: clean up error handling and bare fds by @lmb in https://github.com/cilium/ebpf/pull/1444
• [go.mod] Change go directive to 1.21 by @mx-psi in https://github.com/cilium/ebpf/pull/1441
• gh: don't use code blocks in Bug report template by @lmb in https://github.com/cilium/ebpf/pull/1450
• CI: Update kernel from v6.7 to v6.8 by @dylandreimerink in https://github.com/cilium/ebpf/pull/1452
• build(deps): bump mkdocs from 1.5.3 to 1.6.0 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1455
• build(deps): bump golangci/golangci-lint-action from 4.0.0 to 5.1.0 by @dependabot in https://github.com/cilium/ebpf/pull/1454
• build(deps): bump mkdocs-material from 9.5.16 to 9.5.20 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1456
• build(deps): bump mkdocs-git-revision-date-localized-plugin from 1.2.4 to 1.2.5 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1458
• build(deps): bump jinja2 from 3.1.3 to 3.1.4 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1459
• replace virtme by @lmb in https://github.com/cilium/ebpf/pull/1321
• ci: ignore kprobe_multi test failures on arm64 by @lmb in https://github.com/cilium/ebpf/pull/1461
• perf: flush Record when deadline is exceeded by @lmb in https://github.com/cilium/ebpf/pull/1429
• link: allow retrieving perf event file descriptor by @lmb in https://github.com/cilium/ebpf/pull/1425
• perf_event_open cpu should be -1 for valid PID by @brycekahle in https://github.com/cilium/ebpf/pull/1464
• link: add test suite for netkit by @brlbil in https://github.com/cilium/ebpf/pull/1463
• debug sporadic vimto hang by @lmb in https://github.com/cilium/ebpf/pull/1466
• build(deps): bump requests from 2.31.0 to 2.32.0 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1469
• ci: use stable-selftests tag by @lmb in https://github.com/cilium/ebpf/pull/1472
• link: add Info() method to all link types by @rgo3 in https://github.com/cilium/ebpf/pull/1435
• Add RecursionMisses to access ProgramInfo recursion misses counter by @olsajiri in https://github.com/cilium/ebpf/pull/1465
• elf: turn BTF map_extra into ErrNotSupported by @lmb in https://github.com/cilium/ebpf/pull/1483
• docs: add a section on examples by @lmb in https://github.com/cilium/ebpf/pull/1482
• build(deps): bump mkdocs-material from 9.5.20 to 9.5.26 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1484
• build(deps): bump golangci/golangci-lint-action from 5.1.0 to 6.0.1 by @dependabot in https://github.com/cilium/ebpf/pull/1479
• build(deps): bump mkdocs-git-authors-plugin from 0.8.0 to 0.9.0 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1476
• build(deps): bump mkdocs-git-revision-date-localized-plugin from 1.2.5 to 1.2.6 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1477
• map: fix TestMapBatch failing with ENOSPC for hash maps by @lmb in https://github.com/cilium/ebpf/pull/1485
• gh: try to group dependabot updates by @lmb in https://github.com/cilium/ebpf/pull/1412
• map: add Map.Handle() method for retrieving a Map's type information by @ti-mo in https://github.com/cilium/ebpf/pull/1488
• build(deps): bump urllib3 from 2.2.1 to 2.2.2 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1489
• add example using Linux TC hook for network flow monitoring by @smagnani96 in https://github.com/cilium/ebpf/pull/1352
• docs: fix clipped headline on home page by @lmb in https://github.com/cilium/ebpf/pull/1493
• Fix ringbuffer ring size reporting by @usamasaqib in https://github.com/cilium/ebpf/pull/1492
• Add Flush to manually unblock Read/ReadInto by @brycekahle in https://github.com/cilium/ebpf/pull/1491
• bpf2go: export core functionality by @lmb in https://github.com/cilium/ebpf/pull/1494
• allow to pin kprobe and kprobe_multi links by @olsajiri in https://github.com/cilium/ebpf/pull/1496
• build(deps): bump mkdocs-material from 9.5.26 to 9.5.27 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1498
• link: allow pinning uprobe multi links by @lmb in https://github.com/cilium/ebpf/pull/1499
• program, btf: probe correct log buffer size by @lmb in https://github.com/cilium/ebpf/pull/1500
• build(deps): bump certifi from 2024.6.2 to 2024.7.4 in /docs by @dependabot in https://github.com/cilium/ebpf/pull/1504
• cmd/bpf2go: make Nick Zavaritsky a reviewer by @lmb in https://github.com/cilium/ebpf/pull/1505
• perf: fix panic when CPU is offline by @Ghostbaby in https://github.com/cilium/ebpf/pull/1503
• map: fix flaky TestMapIteratorAllocations by @lmb in https://github.com/cilium/ebpf/pull/1506
• link: fix nil pointer dereference in AttachXDP by @gavinbunney in https://github.com/cilium/ebpf/pull/1507
• info: make it harder to add backwards incompatible API by @lmb in https://github.com/cilium/ebpf/pull/1515
• perf: do not add offline CPUs by @florianl in https://github.com/cilium/ebpf/pull/1510
• fix a variety of Copy() problems by @lmb in https://github.com/cilium/ebpf/pull/1518
• map: do not allocate on lookup when key doesn't exist by @brycekahle in https://github.com/cilium/ebpf/pull/1519
• program: deprecate LogSize and VerifierError.Truncated by @lmb in https://github.com/cilium/ebpf/pull/1520
• btf: fix panic when copying nil Type by @lmb in https://github.com/cilium/ebpf/pull/1521
• map: remove misleading error message when creating without BTF by @lmb in https://github.com/cilium/ebpf/pull/1522

New Contributors

• @mx-psi made their first contribution in https://github.com/cilium/ebpf/pull/1441
• @brlbil made their first contribution in https://github.com/cilium/ebpf/pull/1463
• @smagnani96 made their first contribution in https://github.com/cilium/ebpf/pull/1352
• @usamasaqib made their first contribution in https://github.com/cilium/ebpf/pull/1492
• @Ghostbaby made their first contribution in https://github.com/cilium/ebpf/pull/1503
• @gavinbunney made their first contribution in https://github.com/cilium/ebpf/pull/1507

Full Changelog: https://github.com/cilium/ebpf/compare/v0.15.0...v0.16.0