Sergiu Moga [Wed, 22 Mar 2023 18:57:22 +0000 (20:57 +0200)]
plat/kvm/x86: Make SMP init code resolve its own `start16` relocations
Before waking up the secondary cores, the SMP initialization code copies
the, now position independent through `uk_reloc`, 16-bit and 32-bit
bootstrapping code to a physical page, in the lower 1 Mib of physical
memory. Since, at this point of execution, the immediate values used by
this bootstrapping code take the form of `UK_RELOC_PLACEHOLDER`, they
will need to be properly resolved after being moved into lower memory.
Therefore, add a new locally defined `struct uk_reloc` array holding the
hardcoded corresponding entries of these relocations, adapted to
reference the desired relocation address. Use this array after memory
copying the bootstrapping code to lower memory to resolve its
corresponding `start16` relocations.
Encode these entries with the help of `start16` related macro's and
since these definitions are starting to visually occupy a lot of space,
move everything to `start16_helpers.h`, a separate header file.
Sergiu Moga [Wed, 22 Mar 2023 18:45:52 +0000 (20:45 +0200)]
plat/kvm/x86: Make the Unikernel position independent
Replace all `mov`'s and data declarations (`.long`, `.quad` etc.) that
use absolute symbol references with the equivalent `ur_*` macro's.
One exception to this type of relocation is going to be the 16-bit
code, as there is no point in relocating it before SMP initialization
without even knowing the physical address in the first 1MiB where the
secondary cores are to start execution. So, implement dedicated `ur_*`
macro's whose only use-case is the 16-bit code to cope with its
existence. Furthermore, add an exception to these `start16` relocation
symbols in the script.
Since, on this platform, we make use of static page tables, these were
also made relocatable through the `ur_pte` macro.
plat/kvm/x86: Move `x86_start16_addr` out of `.data.boot`
Since `.data.boot` is meant mainly for boot specific data such as
a Multiboot header, move this unrelated `x86_start16_addr` out of
this section. The reason we do this is because this might cause
conflict with actual boot related data (e.g. `x86_start16_addr` be
placed at the beginning of the binary instead of the actual boot
related header data).
Similarly to the 64-bit C code `do_uk_reloc` self relocator,
`do_uk_reloc32` is a macro that takes as argument a value equal or
different from 0 to specify whether a stack is supplied or not. If the
stack is not supplied, the macro can generate its own minimal scratch
stack. Furthermore, the base virtual address is expected in %esi:%edi
(lower 32 bits in %edi and higher 32 bits in %esi), as well as the
physical base address in the %edx register. Since we are in Protected
Mode, we can assume a 32-bit register is enough to hold this address.
Due to the high number of references to absolute symbol values in the
16-bit and 32-bit bootstrap code, a new macro, `ur_mov`, was introduced.
The new macro creates a new type of `_uk_reloc_` symbol, with an `imm`
suffix, to show that it is placed right after a `mov` instruction whose
immediate value must be relocated/patched. Thus, update `mkukreloc.py`
accordingly.
Furthermore, although useless, for completeness's sake add a 64-bit
variant. This variant does a `movabs` with the placeholder to
guarantee that the size of the immediate is 8 bytes.
Sergiu Moga [Tue, 28 Feb 2023 20:02:04 +0000 (22:02 +0200)]
plat/kvm/arm: Make the Unikernel position independent
Replace the static page table entries with relocatable ones through
the `ur_pte` macro. Thus, we make sure that relocations take place
before enabling the MMU by making a call to `do_uk_reloc` self relocator
method right before jumping to `start_mmu`.
Furthermore, make `Config.uk` select `LIBUKRELOC`.
Sergiu Moga [Tue, 28 Feb 2023 19:54:24 +0000 (21:54 +0200)]
lin/ukreloc: Add `ur_pte` macro to create relocatable PTE's
In order to cope with our static page tables used by the bootstrap code
we need to also be able to relocate our page table entries if we want
to be position independent on platforms that makes use of such page
tables.
This macro makes use of the already existing `ur_data` macro to make the
script create the usual `struct uk_reloc`. However a small modification
to the script makes the script do an additional lookup for corresponding
`pte_attr` symbols that `ur_pte` creates, so that it knows to also add
the page table entry attributes to the final value that the self
relocator uses to properly resolve a relocation.
Sergiu Moga [Tue, 28 Feb 2023 19:03:20 +0000 (21:03 +0200)]
plat/xen/x86: Make the Unikernel position independent
In order to be position independent, `struct uk_reloc` and the self
relocator based off it are used. Thus, the bootstrapping code calls
`do_uk_reloc` as early as possible, by using the page-sized default
stack that the Hypervisor passes onto us, because the actual stack
that we use, is also used by the Hypervisor and any early world switch
may cause the guest to crash.
Furthermore, the obvious absolute symbol references have been replaced
with their instruction pointer relative, position independent,
equivalents. The `r9` register has been used as a scratch register in
this case since, after a brief code analysis, it's been found to not be
used anywhere else during bootstrap.
Sergiu Moga [Wed, 22 Mar 2023 18:51:57 +0000 (20:51 +0200)]
lib/ukreloc: Implement a `struct uk_reloc` based self relocator
Add `do_uk_reloc` a method that parses the `.uk_reloc` section and
applies the relocations accordingly. As arguments, this method receives
a base physical address `r_paddr` for relocations that employ the
`UKRELOC_FLAGS_PHYS_REL` flag and a base virtual address `r_vaddr`
for those that do not want to be relocated against a physical address,
but rather by a given virtual address (e.g. can be used with KASLR).
In ARM's case, the `-fPIC`/`-fPIE`, with the help of `adrp`,
makes the code reference the page-aligned address of the section
and the offset this symbol has in this section where the relocations
are to be applied to and thus the desired relocated value is
obtained. Thus, unlike `x86` that simply does a `%rip` relative
access, ARM ends up dereferencing that in-section offset when trying
to get the runtime value of `__BASE_ADDR` which creates a chicken-egg
problem: we apply the relocations based on the current base address,
but in order to find the current base address we need to relocate
the value that is placed at the address of where the value of the
base address is supposed to be.
We solve this with the help of the `get_rt_addr()` function, which
forces an `adrp`, `add :lo12:` assembly sequence, which leads to
achieving something similar to x86's `%rip` relative addressing.
A case that may explain the need for `r_paddr` and `r_vaddr` would be
a situation where the previous program loader would place us at a random
physical address (r_paddr) and we then want to apply our own virtual
mappings for KASLR (r_vaddr), over the already applied virtual mappings,
if any.
Furthermore, right before applying relocations, the initial, added
through `mkbootinfo.py`, memory region descriptors are also relocated.
This is done by subtracting the initial link time base address, that
has been statically resolved by the linker in `lt_baddr`, and adding
the runtime base address.
Note: `lt_baddr` will generate a relocation so it will be overwritten
by the relocation loop. We want to keep it in case someone may want
to run the relocator multiple times for the same image. So, instead
of slowing down the relocator by checking each relocation against
`lt_baddr`'s address so that we do not overwrite it, simply back
it up in a temporary variable and restore it after relocations are
done.
Sergiu Moga [Wed, 22 Mar 2023 18:52:47 +0000 (20:52 +0200)]
lib/ukreloc: Add inline function to apply a `uk_reloc` entry
Implement a simple inline function to properly apply, depending on the
size of the relocation, a `uk_reloc` entry.
`apply_uk_reloc` may cause `UBSAN` to issue false positives, in
the case of the `x86` architecture which leads to the Unikernel crashing
since the serial console is not initialized yet, as it is mostly intended
to be called within the early self relocator. So make sure that
`UBSAN` does not touch `apply_uk_reloc` because it makes sense, for
`x86`, to make unaligned accesses (especially in Kernel Space) in order
to resolve non-compliant relocations.
Sergiu Moga [Sun, 19 Mar 2023 17:17:08 +0000 (19:17 +0200)]
lib/ukreloc: Add ld script for sections required for a relocatable image
Add a separate Linker Script containing all the required sections
to achieve positional independence and the ability to self relocate.
The `.uk_reloc`, `.dynsym` and `.rela.dyn` sections will be part of the
main data segment and the last two will be stripped in the end, without
causing a hole in memory, due to the fact that `.uk_reloc` already
contains the forced `struct uk_reloc` relocations and, after being
updated, will also contain the `struct uk_reloc` equivalents of
`.rela.dyn`'s entries, which will be of the same size, so the `.bss`
section will stay in place.
The `.dynamic` and `.dynstr` sections become irrelevant in the final image
so we will strip them as well. In order not to cause a memory hole, place
them after `.comment` section.
Sergiu Moga [Wed, 22 Mar 2023 18:53:55 +0000 (20:53 +0200)]
lib: Introduce `libukreloc`
In order to start supporting positional independence we need to be able
to self relocate. Thus, define a new custom relocation related structure
`struct uk_reloc` that is meant to hold the offset in memory where the
relocation is to be applied, the offset of the value from the original
symbol value computed by the linker through the linker script's base
reference address, the size of the relocation and a flags field
respectively (for now we only have the `UKRELOC_FLAGS_PHYS_REL` flag
meant to indicate whether the relocation is to be applied based on a
physical address or not, which comes in handy in bootstrap code).
The final binary blob that the `mkukreloc.py` script builds is made of
a signature (UKRELOC_SIGNATURE), `struct uk_reloc` entries appended
to each other and ends with a zeroed out `struct uk_reloc` to act as a
sentinel. This binary blob will be placed in the binary through the
`.uk_reloc` ELF section, updated through `objcopy`. Therefore, in order
for this forced section update to not mess up linker script exported
symbols, this section must be placed towards the end of the binary.
The binary blob is obtained by parsing all of the `.rela.dyn` entries,
converting them into `struct uk_reloc` entries and by searching
through the debug image's symbols for `_uk_reloc_` symbols created with
the help of the architecture independent helper assembly macro:
`ur_data`. This macro has been added to aid in replacing the references
to absolute symbol values with position independent equivalents that,
instead of referencing this very symbol, it places a placeholder value
(UKRELOC_PLACEHOLDER) and generates a symbol to be parsed by the script
to build a uk_reloc entry. Furthermore, it increases the size of the
`.uk_reloc` section by one entry through the `ur_sec_updt` macro. Note
that `ur_sec_updt` also inserts a dummy relocation entry in `.uk_reloc`
in order to force the linker to not optimize away the symbols. A unique
`uk_reloc` symbol will be generated through the `ur_sym` macro so that
an `ur_*` macro can be used on the same symbol more than once.
Consider the following usage example:
`ur_data quad, symbol, 8, _phys`
This will result in the `nm` tool, whose output the script will parse,
generating such entry:
`0000000000100106 T symbol_uk_reloc_data8_phys`
For `mkukreloc.py` this means, with a linker script reference address
of 0x100000:
```
struct uk_reloc {
__u64 r_mem_off = 0x106; // 0x100106 - 0x100000
__u64 r_addr;
__u32 r_sz = 8; // symbol_uk_reloc_data[8]_phys
__u32 flags = UKRELOC_FLAGS_PHYS_REL; // symbol_uk_reloc_data8[_phys]
} __packed;
```
In order for actual value of `symbol` to be found, since its name is at
the beginning of the generated symbol, the script will now look for such
entry:
`0000000000100078 T symbol` // from [symbol]_uk_reloc_data8_phys
and thus, `0x78` becomes the value of `r_addr` and the
`struct uk_reloc` is completed and appended to the binary blob with the
others.
`mkukreloc.py` will be invoked through `build_uk_reloc` Makefile definition
and the "section.*lma.*adjusted to.*" type of `objcopy` warnings will be
ignored since they are intended and harmless. The script will also check
that the distance between `.uk_reloc` and `.bss` is enough to not cause
an adjustment or spill. In case it is not enough, then the script will
raise an Exception.
All of this functionality shall be contained within the `libukreloc`
library.
Sergiu Moga [Tue, 28 Feb 2023 19:38:34 +0000 (21:38 +0200)]
build: Add configuration option to build the Unikernel as a static PIE
Add compiler and linker options to properly build the Unikernel as a
static PIE. For now, only KVM and Xen are supported.
Strip `*dyn*` related ELF sections from the final binary. This will cause
tools such as `readelf` or `objdump` to issue some warnings or errors
when using on the final image due to the stripped unnecessary sections,
but they will still work and, as a trade off, the image will be as little
as it can be.
Sergiu Moga [Fri, 24 Feb 2023 17:59:50 +0000 (19:59 +0200)]
plat/kvm: Drop `elf32-i386` output target linker flag
Remove the `elf32-i386` linker flag that forces the final Unikernel
binary output as a 32-bit kernel.
Since `qemu` currently boots us through `multiboot`, a boot protocol
that does not support 64-bit kernels, add a new support script
to be run on the final image. This `python` script will convert the
Unikernel's `ELF64` headers to their `ELF32` equivalents, thus fooling
the loader into thinking it's a 32-bit kernel. Since most of the
addresses contained into the fields of the headers are assumed to fit into
at most 4 bytes, the script simply takes those values and directly places
them in their corresponding `ELF32` header fields without truncating.
The actual `ELF32` header will be prepended to the final binary together
with the equivalent `ELF32` Program Headers and the Section Headers will
be appended to the end of the binary so that we can keep the `Multiboot`
header in the first 8192, as the specification requires. Although the
specification does not require the Program Headers be in the first
8192 bytes, GRUB seems to want it to be like that.
The script will be called using a newly added `Makefile.rules` file meant
to contain definitions common to all platforms.
Since, at this time, the only supported boot protocol on x86 KVM QEMU is
`multiboot`, enable this script for x86 KVM QEMU Multiboot builds.
Sergiu Moga [Mon, 15 May 2023 06:44:37 +0000 (09:44 +0300)]
plat/kvm: Compute `ukplat_bootinfo` structure on the final binary
Since it is the final binary that is loaded and executed, make sure
that `build_bootinfo` is called on the final stripped binary, instead
of the debug image, whose segments' sizes may vary after stripping.
support/scripts/mkbootinfo.py: Ensure Program Headers are sorted
Due to the placements of the `tls` and `tls_load` Program Headers,
The last two memory regions inserted in bootinfo structures will
always end up unordered. Thus, make sure they are ordered by their
address first, before building the memory regions and inserting
them.
When the buddy allocator chooses a page order to allocate it can (for
pathologically large values of num_pages) start from a value larger than
FREELIST_SIZE, leading to out-of-bounds access of the freelist array.
This change hardens the out-of-memory check to prevent this.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Eduard-Florin Mihailescu <mihailescu.eduard@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #932
This adds a Kconfig option to enable sanity checking of buddy allocator
free lists at runtime. Check points are placed at the beginning and end
of functions that operate on the free lists. When the option is disabled
the checks default to zero-overhead no-ops.
In addition, this adds an assert that returned allocated pages are
correctly aligned to their buddy order.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Eduard-Florin Mihailescu <mihailescu.eduard@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #932
Simon Kuenzer [Tue, 8 Aug 2023 12:08:23 +0000 (14:08 +0200)]
Makefile: Include Makefile.build from main Makefile
This commit ensures that the inclusion of sub-'Makefile.build' is done with
the main Makefile. This is done for consistency reasons.
This basically adopts
commit db20b80c47b2 ("Makefile: Allow external Makefile.build")
from GitHub PR #1005.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #1028
Marco Schlumpp [Tue, 16 May 2023 14:45:49 +0000 (16:45 +0200)]
plat/kvm: Add ectx assertion in interrupt handler
This checks that the extended registers were not modified within the
interrupt service routine. This is not free and therefore can be
toggled via a KConfig option.
Signed-off-by: Marco Schlumpp <marco@unikraft.io> Reviewed-by: Razvan Deaconescu <razvand@unikraft.io> Approved-by: Simon Kuenzer <simon@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #897
Marco Schlumpp [Tue, 16 May 2023 14:43:21 +0000 (16:43 +0200)]
arch/x86: Add function to check for unmodified ectx
This can be used in situtations where the executed code must not touch
the extended registers. For example, interrupt handlers in Unikraft do
save these and therefore also cannot modify them. Doing so is usually
breaking the interrupted code.
Signed-off-by: Marco Schlumpp <marco@unikraft.io> Reviewed-by: Razvan Deaconescu <razvand@unikraft.io> Approved-by: Simon Kuenzer <simon@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #897
Now that we can dynamically mount any kind of volume through a more
generic format through the commandline, there is an unnecessary
overlap between `vfs.{rootfs, rootdef, rootops,rootflags}` and
defining `rootfs` related arguments through `vfs.fstab` volumes.
Therefore, deprecate `vfs.{rootfs, rootdef, rootops,rootflags}` in
favor of `vfs.fstab`.
Signed-off-by: Sergiu Moga <sergiu@unikraft.io> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Reviewed-by: Razvan Virtan <virtanrazvan@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Reviewed-by: Simon Kuenzer <simon@unikraft.io> Approved-by: Simon Kuenzer <simon@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #979
Provide a command-line argument `vfs.fstab` that is meant to
contain a list of whitespace separated strings with the following
format:
```
vfs.fstab=[
"<src_dev>:<mntpoint>:<fsdriver>[:<flags>:<opts>]"
"<src_dev>:<mntpoint>:<fsdriver>[:<flags>:<opts>]"
...
]
```
The core will parse the provided strings as volume information and
attempt to mount them accordingly, if `CONFIG_LIBVFSCORE_FSTAB` is
provided.
Furthermore, add a configurable `CONFIG_LIBVFSCORE_FSTAB_SIZE`
option that will be used as the maximum amount of automatically mounted
volumes, excluding the `rootfs` and modularize `initrd` extraction and
mounting functionality into a standalone function to be used by both
`fstab` and implicit `rootfs`.
As a last change, since the file's functionality drastically changed,
rename `rootfs.c` to `automount.c`.
Signed-off-by: Sergiu Moga <sergiu@unikraft.io> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Reviewed-by: Razvan Virtan <virtanrazvan@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Reviewed-by: Simon Kuenzer <simon@unikraft.io> Approved-by: Simon Kuenzer <simon@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #979
Enclose parameters of function-like macros in brackets in x86_64 paging
code to avoid potential unwanted operator precedence consequences after
macro expansion.
Alexander Jung [Wed, 9 Aug 2023 13:08:57 +0000 (13:08 +0000)]
.github/workflows: Fix missing dependency to `LIBCOMPILER_RT`
Following the merge of [0], the use of complex numbers requires the
external microlibrary `compiler-rt`[1]. Without this inclusion, builds
that set `CONFIG_LIBMUSL_COMPLEX` will fail.
Andrei Tatar [Wed, 31 May 2023 13:32:31 +0000 (15:32 +0200)]
plat/drivers/virtio: Fix missing lcpu.h include
This change adds an #include of <uk/plat/lcpu.h> which provides
`ukplat_lcpu_disable_irq()` and `ukplat_lcpu_enable_irq()`, that are
used by `virtio_blkdev_queue_cleanup_requests()`.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #925
Marco Schlumpp [Tue, 18 Apr 2023 07:39:39 +0000 (09:39 +0200)]
lib/vfscore: Improve and fix trace point format
There were multiple issues with the format string used in the trace
points. For example, python, which is used to interpret the resulting
trace files, does not support the `%p` specifier, and we instead have to
use `%x`. Also, we can make use of the alternate format (e.g. `%#x`) to
ensure the hexadecimal numbers have a `0x` prefixed. Some format strings
also did not use the correct number of arguments.
Marco Schlumpp [Tue, 16 May 2023 14:52:24 +0000 (16:52 +0200)]
build: Use -nostdinc for clang
This flag ensures that no system header directories are used by the
compiler. If this flag is not used, the compiler could silently fall
back to system headers instead of complaining about not finding a
header.
Signed-off-by: Marco Schlumpp <marco@unikraft.io> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #895
Simon Kuenzer [Fri, 30 Jun 2023 12:23:15 +0000 (14:23 +0200)]
lib/ukschedcoop: Keep track of thread execution time
This commit introduces an implementation for tracking thread execution
time with minimal execution overhead. On every scheduler call
(`uk_sched_yield()`), the execution time of the current thread is updated.
`__nsec` and `__snsec` are defined as 64-bit integers based on `__u64` and
`__s64`. Depending on the target architecture, these data types are based
on either `long` or `long long`. The time conversion macros, such as
`ukarch_time_nsec_to_sec()`, did not strictly adhere to the data type
definition, as some basic constants were defined as `UL`, while others
were defined as `ULL`, resulting in promotion to `long long` when the
architecture used `long` for 64-bit values. Although `long long` is
effectively treated as a 64-bit integer by `gcc`, the compiler still
issues a warning for mismatched integer types when the print format
definitions `__PRInsec` and `__PRIsnec` are used and a promotion had
occurred.
Simon Kuenzer [Tue, 8 Aug 2023 13:27:40 +0000 (15:27 +0200)]
lib/uklibid: Supporting pre-built libraries
This commit introduces a clearer separation between runtime and
compile-time resolution of library identifiers with the API. This is done
so that libraries can potentially be built off-tree and shipped in binary
form. Each library's own identifier (`uk_libid_self()`) is stored in a
constant variable that is resolved during link-time. The owner of this
variable is `uklibid`, so the identifier is external to a binary library.
A binary delivery of a library can be created with Unikraft (for example:
`make libbinary`) and the resulting library object (likely under
`build/libbinary.o`) can be delivered with a `Config.uk` like:
```KConfig
config LIBBINARY
bool "Binary example library"
default n
```
and a `Makefile.uk` like:
```Makefile
UK_OLIBS-$(CONFIG_LIBBINARY) += $(IMPORT_BASE)/libbinary.o
UK_LIBID_EXTRA-$(CONFIG_LIBBINARY) += libbinary
```
Please note that the library must be registered directly in `UK_OLIBS-y`;
`addlib` or `addlib_s` cannot be used in such cases.
Please also note that such a library should restrict dependencies on other
libraries and their configuration options. In particular, the use of macros
and inline functions must be handled with care.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Robert Kuban <robert.kuban@opensynergy.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #938
Simon Kuenzer [Fri, 2 Jun 2023 00:21:25 +0000 (02:21 +0200)]
lib/ukstore: Re-use `libraries.in` from `lib/uklibid`
Instead of generating own library identifiers, this commit adopts
`lib/ukstore` to use the same library catalog from `lib/uklibid`. At the
same time, this commit adopt `Makefile.uk` to make use of the `SUBBUILD`
feature of the build system. This is done for simplifying Makefile.uk and
to let the build system natively generate each dynamic headers of
`lib/ukstore`.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Robert Kuban <robert.kuban@opensynergy.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #938
Simon Kuenzer [Wed, 31 May 2023 15:44:18 +0000 (17:44 +0200)]
lib/ukdebug: Use lib/uklibid for library names
Every invocation of any of the provided print function macros by
`lib/ukdebug` (`uk_printk()`, `uk_printd()`), caused placing a C-string
containing the library name in the sources. This commit replaces this
C-string with a library identifier which is just an unsigned integer. The
ID is only resolved into a name in the actual print format function
(`_vprint()`). The resolution is done by `uk_libname()` that is using the
ID for a table lookup (O(1)).
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Robert Kuban <robert.kuban@opensynergy.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #938
Simon Kuenzer [Wed, 31 May 2023 16:27:56 +0000 (18:27 +0200)]
lib/uklibid: Run-time name resolution of library IDs
This commit introduces the function `uk_libname()` that resolves a library
ID to its library name at run-time. The function returns a reference to a
corresponding C-string with a lookup table.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Robert Kuban <robert.kuban@opensynergy.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #938
Simon Kuenzer [Wed, 31 May 2023 16:19:56 +0000 (18:19 +0200)]
lib/uklibid: Compile-time mapping of library names to IDs
This commit introduces macros for mapping a library name to a compile-time
unique identifier with the pre-processor. The library IDs are starting from
`0` and increase linearly. The purpose is that such an ID can be directly
used as index for looking up a corresponding entry within an array.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Robert Kuban <robert.kuban@opensynergy.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #938
Simon Kuenzer [Thu, 30 Mar 2023 16:32:03 +0000 (18:32 +0200)]
lib/syscall_shim: Utilize `SUBBUILD` feature within `Makefile.uk`
This commit utilizes the recently added `SUBBUILD` feature of the build
system to natively generate all dynamic headers of the library. This has
several reasons:
1) The `Makefile.uk` of `syscall_shim` becomes more readable
2) No custom rule for creating the subdirectory structure is needed
3) No custom `awk`-rules are needed for generating the headers
4) The build system can take care of handling build order dependencies
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Robert Kuban <robert.kuban@opensynergy.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #938
Simon Kuenzer [Tue, 8 Aug 2023 23:00:52 +0000 (01:00 +0200)]
build: Provide $(IMPORT_BASE) to `Makefile.uk` files
This commit makes the variable `$(IMPORT_BASE)` usable in `Makefile.uk`
files. This is needed to include libraries that do not use the `addlib`
or `addlib_s` registration functions. An example are libraries that are
shipped in binary form.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Robert Kuban <robert.kuban@opensynergy.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #938
Simon Kuenzer [Thu, 1 Jun 2023 12:36:30 +0000 (14:36 +0200)]
build: Introduce `*_SUBBUILD` for compilation units
This commit adds to the build system the ability to place generated
files (e.g., pre-processed, compiled) in a subdirectory of the
corresponding library build directory (instead of just placing them in the
top-level directory). For this purpose, the build system looks for a
file-scoped variable with the suffix `_SUBBUILD`. Such a variable can
contain a path that is created below the library build directory.
Example (`Makefile.uk`):
# Create `myheader.h` under `build/libmylib/include/uk/`
LIBMYLIB_SRCS-y += $(LIBMYLIB_BASE)/myheader.m4>.h
LIBMYLIB_MYHEADER_SUBBUILD += include/uk
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Robert Kuban <robert.kuban@opensynergy.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #938
Simon Kuenzer [Thu, 1 Jun 2023 12:33:30 +0000 (14:33 +0200)]
build: Only first-level subdirectories with `mk_sub_build_dir`
We have two pairs of functions that are internally used to create the
directory structure the build output directory (normally `build/`):
- The function `mk_sub_build_dir` is intended to create only the first
level subdirectory under the build output directory for a library.
`sub_build_dir` is a companion function that returns the full path
of such a first-level subdirectory for a given library name.
- The function `mk_sub_libbuild_dir` is intended to create further
subdirectories under a library build directory that was created by
`mk_sub_build_dir`. This function allows multiple levels of
subdirectories (example: to create an header include folder:
'include/uk/bits').
`sub_libbuild_dir` is the companion function that returns the
full path of such a library subdirectory.
This commit corrects the implementation so that this behavior is
enforced.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Robert Kuban <robert.kuban@opensynergy.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #938
Simon Kuenzer [Thu, 1 Jun 2023 10:52:16 +0000 (12:52 +0200)]
build/Makefile.rules: Improve parameter description of functions
Most of our Makefile commands have a preceding usage description in the
form of:
# func_name $param_desc,$param_desc,...
Because most of these functions reference the parameters by their
position, this commits changes the description to:
# func_name $1:param_desc,$2:param_desc,...
This should simplify the reading of Makefile function definitions.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Robert Kuban <robert.kuban@opensynergy.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #938
Andrei Tatar [Mon, 24 Jul 2023 15:14:15 +0000 (17:14 +0200)]
lib/ukcpio: Fix symlinks not being extracted
Previously the CPIO extraction code would only handle sections whose
header was of type directory or file, skipping symlinks.
This change adds code to handle extracting symlinks from CPIO archives,
as well as a warning message for unknown CPIO sections.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #992
Andrei Tatar [Mon, 24 Jul 2023 16:29:34 +0000 (18:29 +0200)]
lib/vfscore: Accept negative timestamps
Previously vfs code would reject timestamps that are more than 1 second
before the epoch as invalid. This is wrong; unix time is defined as a
signed offset and as such can and must support dates before the epoch.
This change relaxes the overly-strict validity check, allowing the full
range of valid file timestamps to be used.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #991
Andrei Tatar [Tue, 20 Jun 2023 11:30:32 +0000 (13:30 +0200)]
lib/vfscore: Fix behavior of writable open on dirs
Previously the open syscall, when called on a directory with write mode
would either fail with EINVAL or, worse, finish successfully, whereas
the expected behavior is to return EISDIR.
This change corrects this behavior, bringing open() in line with
expectations.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Tu Dinh Ngoc <dinhngoc.tu@irit.fr> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #953
Andrei Tatar [Tue, 20 Jun 2023 11:19:42 +0000 (13:19 +0200)]
lib/vfscore: Fix double-lock bug in symlink
This change fixes an issue where the lock of the destination parent
directory was being taken more than once in the symlink syscall by using
the locked variant of namei_last_nofollow.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #952
Andrei Tatar [Tue, 20 Jun 2023 11:17:06 +0000 (13:17 +0200)]
lib/vfscore: Add locked namei_last_nofollow
This change adds a variant of namei_last_nofollow that can be called
with the lock on ddp->d_vnode held, as well as factoring out common code
between the two function variants.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #952
Andrei Tatar [Tue, 20 Jun 2023 10:13:08 +0000 (12:13 +0200)]
lib/vfscore: Reorder locks in *at syscalls
*at syscalls are implemented in Unikraft in two stages: (1) build the
full target path relative to the supplied dirfd, and (2) delegate to the
regular syscall. Previously these syscalls held the lock on the dirfd
vnode throughout steps (1) and (2); however, the sub-syscalls often need
to acquire the same lock (along with perhaps others), causing
double-lock asserts to trigger.
This commit changes this logic to only hold the vnode lock during step
(1) and release it immediately before (2). While this relaxes the
previous atomicity guarantees somewhat, holding the vnode lock on dirfd
only prevented concurrent changes to first-generation descendents, and
unorthogonal and arguably unexpected behavior. The useful purpose of
holding the dirfd lock is to ensure we build a full path that was (at
one point in time) valid, with the sub-syscalls implementing their own
atomicity guarantees as per usual.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #951
Andrei Tatar [Wed, 7 Jun 2023 10:18:46 +0000 (12:18 +0200)]
lib/posix-sysinfo: Provide _SC_GETPW_R_SIZE_MAX
Calling code expects sysconf(_SC_GETPW_R_SIZE_MAX) to be either -1 or a
non-zero value for an initial buffer size. The previous default value of
0 is outside of specifications and can cause unintended behavior.
This change makes sysconf(_SC_GETPW_R_SIZE_MAX) return -1.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Ioan-Teodor Teugea <ioan_teodor.teugea@stud.acs.upb.ro> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #936
Stefan Jumarea [Sun, 4 Jun 2023 18:57:50 +0000 (21:57 +0300)]
lib/ukdebug: Add `LVLC_THREAD` macro
If `LIBUKDEBUG_PRINT_THREAD` is selected in the configuration menu and
`LIBUKDEBUG_ANSI_COLOR` is not, a build error is thrown because the
`LVLC_THREAD` macro is not set.
Fix this by setting the macro to an empty string if `LIBUKDEBUG_ANSI_COLOR`
is not set.
Signed-off-by: Stefan Jumarea <stefanjumarea02@gmail.com> Reviewed-by: Eduard-Florin Mihailescu <mihailescu.eduard@gmail.com> Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #929
Andrei Tatar [Wed, 31 May 2023 11:40:08 +0000 (13:40 +0200)]
arch/x86: Silence warning about incomplete switch
Commit ffdbb486c4dd ("arch/*: Add sanitization function for ectx memory")
added a function for sanitizing ECTX memory in cases where it is needed.
It also triggers a warning about the switch not handling all values.
This change silences this warning by adding a no-op default.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Eduard Vintilă <eduard.vintila47@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #926
Andrei Tatar [Mon, 24 Jul 2023 15:39:35 +0000 (17:39 +0200)]
lib/ramfs: Implement new file timestamps
This change implements setting the file times of newly created files to
the current time, filling in a placeholder that previously would set the
times to the epoch (i.e., 1970-01-01 00:00 UTC).
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Delia Pavel <delia_maria.pavel@stud.acs.upb.ro> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #995
Andrei Tatar [Mon, 24 Jul 2023 15:30:24 +0000 (17:30 +0200)]
lib/ramfs: Ensure stability of inode numbers
Previously ramfs would assign a new unique inode number to a file on
every successful filesystem lookup. This can lead to the same file being
allocated multiple different inodes, breaking code that (correctly) uses
inodes as reliable indicators of file identity.
This change makes inode numbers persistent, allocated along with the
filesystem node, and stable across multiple lookups of the same file.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #996
Andrei Tatar [Mon, 24 Jul 2023 15:21:33 +0000 (17:21 +0200)]
lib/vfscore: Fix create through broken symlink
Previously a vfs syscall on a broken symlink would fail with ENOENT
instead of attempting the syscall on the symlink target. This would make
open(O_CREAT) to fail instead of creating the missing target file.
This change fixes this behavior, bringing it in line with expectations.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #998
Marc Rittinghaus [Thu, 20 Oct 2022 08:42:21 +0000 (10:42 +0200)]
Makefile: Change gccgo-7 to gccgo
The current Makefile refers to gccgo via gccgo-7. This commit removes
the version suffix.
Co-authored-by: Marc Rittinghaus <marc.rittinghaus@unikraft.io> Signed-off-by: Marc Rittinghaus <marc.rittinghaus@unikraft.io> Signed-off-by: Eduard Vintilă <eduard.vintila47@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #1005
The current method of building libraries is to iterate over
each library's source files and invoke a file type specific
build rule. For libgo this is insufficient as we have to do
more complex operations for dependency resolution
before starting the actual build. This commit allows libraries
to supply an own `Makefile.build` that enables custom
build mechanics.
Co-authored-by: Marc Rittinghaus <marc.rittinghaus@unikraft.io> Signed-off-by: Marc Rittinghaus <marc.rittinghaus@unikraft.io> Signed-off-by: Eduard Vintilă <eduard.vintila47@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #1005
plat/common: Add support for pvpanic devices for x86
Unikraft already offers exit codes through the ISA device, but these
are only a convention and are not understood by QEMU itself.
When using `-device pvpanic`, a GUEST_PANICKED event[1] is sent to QEMU
when the unikernel crashes.
This event can be seen when using QMP[1] for example.
The bits corresponding to each even are set as per the documentation[2].
The main usage of this is in kraftkit where unikernels are detached
and exit codes are lost.
Using this can signal when the unikernel crashes.
Andrei Tatar [Mon, 24 Jul 2023 16:12:06 +0000 (18:12 +0200)]
lib/vfscore: Fix wrong application of umask
Previously vfscore would apply the umask to the mode bits in the
emulated libc functions `open` and `openat`, instead of within the
open/openat syscall implementation where it would be correct to do so.
This lead to the umask being ignored when a real libc was in use.
This change fixes this bug, restoring expected behavior with or without
a libc.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Reviewed-by: Eduard Vintilă <eduard.vintila47@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #990
Andrei Tatar [Mon, 24 Jul 2023 16:25:54 +0000 (18:25 +0200)]
lib/vfscore: Correct behavior of utime(NULL)
Previously, as a placeholder behavior, utime(NULL) would set the file
times to the epoch (1970-01-01 00:00 -0000) instead of the current time.
This change corrects this oversight, bringing the syscall in line with
expected and documented behavior.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Eduard Vintilă <eduard.vintila47@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #993
Andrei Tatar [Mon, 24 Jul 2023 16:20:43 +0000 (18:20 +0200)]
lib/vfscore: Add unlinkat syscall
Unikraft was previously missing an implementation of `unlinkat`, a
syscall often used by modern runtimes and libcs in lieu of the older
`unlink` and `rmdir` for deleting files and directories.
This change provides an implementation of `unlinkat` based off existing
syscalls, in the same vein as other `*at` syscalls.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Delia Pavel <delia_maria.pavel@stud.acs.upb.ro> Reviewed-by: Adina-Maria Vaman <adinamariav22@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #994
Andrei Tatar [Mon, 24 Jul 2023 16:07:06 +0000 (18:07 +0200)]
lib/ramfs: Support setting file mode on creation
Previously any file created on ramfs would have the mode bits set to
0777, regardless of the mode bits supplied to open/creat/mkdir, which
can break software that explicitly checks these bits.
This change corrects this oversight, having new files be created with
the requested mode bits set.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #997
Martin Kröning [Wed, 28 Jun 2023 08:35:48 +0000 (10:35 +0200)]
build: Add `UK_LDEPS`
This adds support for the `UK_LDEPS` environment variable.
This variable specifies external files that are watched for changes by make.
If any of the specified files change, the final linking step is redone.
The files are only watched and not consumed to avoid changing the order of supplied `LDFLAGS`.
This was inspired by `cargo:rerun-if-changed=PATH`:
- https://doc.rust-lang.org/cargo/reference/build-scripts.html#rerun-if-changed
Signed-off-by: Martin Kröning <martin.kroening@eonerc.rwth-aachen.de> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Reviewed-by: Marco Schlumpp <marco@unikraft.io> Approved-by: Simon Kuenzer <simon@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #957
Martin Kröning [Fri, 23 Jun 2023 09:38:30 +0000 (11:38 +0200)]
build: Allow setting flags from the environment
This adds `UK_ASFLAGS`, `UK_CFLAGS`, `UK_CXXFLAGS`, `UK_GOCFLAGS`, and
`UK_LDFLAGS` for explicitly setting these build flags for Unikraft.
For reference, see
- https://github.com/torvalds/linux/blob/v6.4/Documentation/kbuild/kbuild.rst#environment-variables
- https://github.com/torvalds/linux/blob/v6.4/Makefile#L1097-L1101
Signed-off-by: Martin Kröning <martin.kroening@eonerc.rwth-aachen.de> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Reviewed-by: Marco Schlumpp <marco@unikraft.io> Approved-by: Simon Kuenzer <simon@unikraft.io> Tested-by: Unikraft CI <monkey@unikraft.io>
GitHub-Closes: #957
Marco Schlumpp [Thu, 3 Aug 2023 09:17:06 +0000 (11:17 +0200)]
plat/*: Signal shutdown cause on QEMU
This uses the `isa-debug-exit` device to pass an exit code to QEMU. By
default this device is not available but can be enabled using the
`-device isa-debug-exit` switch on the QEMU command-line.
projects / unikraft / unikraft.git / log