Andrei Tatar [Wed, 29 May 2024 12:13:23 +0000 (14:13 +0200)]
lib/ukfile: Ensure finalizers run after destructor
This change ensures that file finalizers are executed after the main
file destructor when the last strong reference to a file is released.
Finalizers may themselves release weak references, which in turn may
trigger the file destructor. Previously this could lead to destructors
being called in the wrong order.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Delia Pavel <delia_maria.pavel@stud.acs.upb.ro> Reviewed-by: Eduard Vintilă <eduard.vintila47@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1419
Andrei Tatar [Wed, 17 Apr 2024 17:12:02 +0000 (19:12 +0200)]
lib/posix-fdio: Add bincompat support for RWF_*
This change adds values for RWF_* flags in posix-fdio, allowing it to
interpret their meaning even without support from our (no)libc.
This enhances binary compatibility.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Mihnea Firoiu <mihneafiroiu0@gmail.com> Reviewed-by: Robert Zamfir <georobi.016@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1394
Andrei Tatar [Tue, 7 May 2024 16:33:53 +0000 (18:33 +0200)]
lib/posix-fdio: Allow owner/group == -1 for fchown
This change adds support in fchown for the owner or group to be passed
as -1, in which case that particular field is left unchanged.
This mimimcs the behavior of Linux.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1416
Andrei Tatar [Thu, 1 Feb 2024 17:05:43 +0000 (18:05 +0100)]
lib/posix-tty: Add core tty ioctls to serial files
This change adds support for essential tty-specific ioctl commands to
the serial file implementation of `ctl`. These operations are either
no-ops or return a sensible description of the properties of the serial
file.
Checkpatch-Ignore: ENOSYS Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Marco Schlumpp <marco@unikraft.io> Reviewed-by: Delia Pavel <delia_maria.pavel@stud.acs.upb.ro> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1226
Andrei Tatar [Tue, 16 Jan 2024 14:44:19 +0000 (15:44 +0100)]
lib/posix-tty: Add stat support to tty files
This change adds support to tty files for the stat family of syscalls.
Returned values are a subset of what Linux provides, missing extended
attributes as well as timestamps. Where applicable, values match those
returned by Linux.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Marco Schlumpp <marco@unikraft.io> Reviewed-by: Delia Pavel <delia_maria.pavel@stud.acs.upb.ro> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1226
Andrei Tatar [Thu, 21 Dec 2023 20:32:43 +0000 (22:32 +0200)]
lib/*: Move stdio out of vfscore into posix-tty
This change moves stdio initialization from vfscore into posix-tty,
replacing the legacy stdin/out/err files with newvfs versions.
In addition, this move allows differing file types, either pseudofiles
or serial console, to be assigned independently to stdin and stdout/err.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Marco Schlumpp <marco@unikraft.io> Reviewed-by: Delia Pavel <delia_maria.pavel@stud.acs.upb.ro> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1226
Andrei Tatar [Thu, 21 Dec 2023 20:20:20 +0000 (22:20 +0200)]
lib/posix-tty: Introduce posix-tty library
This change introduces the posix-tty library, tasked with implementing
newvfs files for use as standard in/out/err.
The initial implementation provides drivers for pseudo-files (null,
void, and zero) as well as platform-specific serial console, akin to the
stdio submodule of legacy vfscore.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Marco Schlumpp <marco@unikraft.io> Reviewed-by: Delia Pavel <delia_maria.pavel@stud.acs.upb.ro> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1226
Sergiu Moga [Sun, 18 Feb 2024 09:22:10 +0000 (11:22 +0200)]
plat/kvm/x86: Add early COM1 init/print for CPU init errors
Usually early boot failures tend to be very confusing since there
is no message printed. To ease figuring out what went wrong, implement
a very basic early initialization macro for the COM1 port as well as
a corresponding printing macro that can be used before having a stack.
As a first use case of these newly added macros, print an error message
when failing early CPU features initialization, right before halting the
system.
Andrei Tatar [Thu, 22 Feb 2024 18:08:56 +0000 (19:08 +0100)]
lib/posix-socket: Expose internal socket syscalls
This change exposes Unikraft-internal syscalls that create sockets.
Both versions returning raw uk_files as well as opened file descriptors
are provided.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Mihnea Firoiu <mihneafiroiu0@gmail.com> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1337
Andrei Tatar [Wed, 22 May 2024 12:09:40 +0000 (14:09 +0200)]
lib/posix-unixsocket: Add address sendmsg support
This change adds support for specifying a destination address in a
`sendmsg` call to a connection-free unix socket. The address is looked
up the same as would be done for `connect`.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Delia Pavel <delia_maria.pavel@stud.acs.upb.ro> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1421
Andrei Tatar [Wed, 22 May 2024 12:05:25 +0000 (14:05 +0200)]
lib/posix-unixsocket: Fix mismatched locks
This change fixes a lock/unlock pair with mismatched files in `sendmsg`,
probably introduced by a typo, preventing both crashes and inconsistent
lock state.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Delia Pavel <delia_maria.pavel@stud.acs.upb.ro> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1421
Andrei Tatar [Thu, 22 Feb 2024 17:12:54 +0000 (18:12 +0100)]
lib/posix-timerfd: Replace time syscalls
This change replaces the use of userspace time syscalls in posix-timerfd
with calls to Unikraft-internal syscalls, eliminating an undeclared
dependency on syscall-shim.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Mihnea Firoiu <mihneafiroiu0@gmail.com> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1336
Andrei Tatar [Thu, 22 Feb 2024 17:07:21 +0000 (18:07 +0100)]
lib/posix-time: Add internal syscall interface
This change adds Unikraft-internal syscalls (uk_sys_*) to posix-time,
allowing the use of time functions without either a libc or
syscall-shim selected.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Mihnea Firoiu <mihneafiroiu0@gmail.com> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1336
As agreed among the Unikraft maintainers, this commit removes the Linux
userspace platform target (incuding the tap netdev driver), that was
originally intended for debugging purposes. As there are ongoing efforts
in the Unikraft community to drastically improve the debugging experience
on all hypervisor platforms, there is no good reason to keep the
maintenance effort for the linuxu platform.
This platform already had a large backlog of features.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Marco Schlumpp <marco@unikraft.io> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1422
Andrei Tatar [Wed, 17 Apr 2024 13:04:49 +0000 (15:04 +0200)]
lib/posix-fdio: Support VA args for all fcntl cmds
This change adds support to the `fcntl` libc wrapper for fetching the
optional argument for all known fcntl cmd values.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Robert Zamfir <georobi.016@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Simon Kuenzer <simon@unikraft.io>
GitHub-Closes: #1392
Andrei Tatar [Wed, 17 Apr 2024 12:44:38 +0000 (14:44 +0200)]
lib/posix-fdio: Clean up pread/pwrite aliasing
This change reworks the libc function aliasing for pread(64) and
pwrite(64), simplifying it.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Robert Zamfir <georobi.016@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Simon Kuenzer <simon@unikraft.io>
GitHub-Closes: #1392
Andrei Tatar [Wed, 17 Apr 2024 12:20:10 +0000 (14:20 +0200)]
lib/posix-fdio: Move over libc funcs from vfscore
This change moves the implementations of non-trivial libc wrapper
functions for file-related syscalls from vfscore into posix-fdio, where
these syscalls are actually implemented.
This was an oversight of the original posix-fdio work.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Robert Zamfir <georobi.016@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Simon Kuenzer <simon@unikraft.io>
GitHub-Closes: #1392
Andrei Tatar [Mon, 22 Jan 2024 14:37:30 +0000 (15:37 +0100)]
lib/posix-unixsocket: Add warning for 0-len dgrams
This change adds a warning to the send operation of unixsockets when a
datagram of zero length is attempted to be sent.
Currently unixsockets do not support 0-length datagrams and will
otherwise silently drop these packets. This is due to internal
implementation details that can be addressed when (and if) 0-length
unixsocket datagrams are relied on by workloads.
The warning then serves as a compatibility reminder in misbehaving apps.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1272
Andrei Tatar [Thu, 22 Feb 2024 12:55:39 +0000 (13:55 +0100)]
lib/posix-pipe: Implement O_DIRECT (packet) pipes
This change replaces the implementation of pipe buffers with one that
supports both stream- and packet-mode communication, selected by using
the O_DIRECT flag.
Previously, this had been stubbed as an internal property of pipes, a
design that breaks the separation between files and open file
descriptions. This is now corrected, updating the API and its sole
consumer, posix-unixsocket.
Andrei Tatar [Thu, 7 Mar 2024 20:28:19 +0000 (21:28 +0100)]
include/uk: Add raw key RB trees in tree.h
This change adds the possibility to generate a RB tree that performs
lookups using raw keys instead of full-fledged tree nodes for
comparisons. This is achieved by providing both cmp and key functions.
- key(node) -> key_type
- cmp(key_type, key_type) -> int
The API is left compatible with the old approach using an implicit
identity key function, and where key_type is the same as node.
Checkpatch ignores to maintain consistent style within the file.
Marco Schlumpp [Mon, 24 Apr 2023 15:26:10 +0000 (17:26 +0200)]
include/uk: Add splay/RB tree implementation from FreeBSD
These can be used to implement ordered collections of structures.
Taken from FreeBSD 13.3.0 with the following modifications:
- Unikraft header guard format (__UK_TREE_H__)
- replaced FreeBSD types with Unikraft-internal
- prefixed all macros with UK_
Checkpatch ignores to leave code as close to upstream as possible.
Andrei Tatar [Wed, 7 Feb 2024 15:28:56 +0000 (16:28 +0100)]
lib/posix-unixsocket: Add basic *sockopt support
This change adds getsockopt/setsockopt support for basic socket options
from the SOL_SOCKET family. There are two main types of options added:
- Read-only opts about socket state (e.g. SO_ACCEPTCONN)
- No-op opts for benign unsupported features
Checkpatch-Ignore: ENOSYS Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Eduard Vintilă <eduard.vintila47@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1314
Andrei Tatar [Thu, 8 Feb 2024 16:44:13 +0000 (17:44 +0100)]
lib/posix-poll: Add option to yield on wait
This change adds a Kconfig option, LIBPOSIX_POLL_YIELD, that when set
ensures that execution is yielded at the beginning of every call to
epoll_wait (as well as select and poll).
This can aid compatibility with apps that assume a starvation-free
scheduler.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Mihnea Firoiu <mihneafiroiu0@gmail.com> Reviewed-by: Radu Nichita <radunichita99@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1319
Andrei Tatar [Wed, 17 Apr 2024 17:25:23 +0000 (19:25 +0200)]
lib/ukfile: Add utility inlines for iovec I/O
This change adds a utility header providing convenience inlines for
doing I/O on buffers described by struct iovec, namely:
- zero out
- scatter data from buffer to iov
- gather data from iov into buffer
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Eduard Vintilă <eduard.vintila47@gmail.com> Approved-by: Marco Schlumpp <marco@unikraft.io>
GitHub-Closes: #1396
Andrei Tatar [Fri, 23 Feb 2024 13:55:38 +0000 (14:55 +0100)]
lib/ukfile: Add opt-in support for file finalizers
This change adds optional support for file finalizers -- custom
functions registered to run when the last strong reference to a file is
released. These can be useful for e.g., automatically removing a closed
file from a polling pool.
Since this feature adds some overhead and may not be always required, it
is gated behind the LIBUKFILE_FINALIZERS config option. With this option
in its default disabled state, behavior and mem usage is as before.
This commit changes the driver API of ukfile, specifically its refcount
initializers. Affected consumers of ukfile have also been patched.
Checkpatch-Ignore: MACRO_ARG_REUSE
Checkpatch-Ignore: TRAILING_STATEMENTS Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Maria Pana <maria.pana4@gmail.com> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1341
Andrei Tatar [Thu, 14 Mar 2024 15:44:48 +0000 (16:44 +0100)]
lib/ukcpio: Overwrite destination if exists
This change adds logic to CPIO extraction that attempts to remove or
rename an existing destination path. Specifically:
- regular files and symlinks will be unlinked
- empty directories will be removed
- non-empty directories will be renamed to NAME.0
When extracting a directory on top of an existing directory, the latter
is not removed or renamed, and only has mode bits adjusted.
We choose renaming over recursive deletion because:
- replacing directories with other files should ideally be a rare event
- recursive deletion, while storage efficient, is nontrivial and costly
to perform; renaming OTOH is fast but wasteful
- we value boot latency in Unikraft, thus picking rename
- this tradeoff should be revisited if/when either (1) we have efficient
recursive directory removal or (2) we value storage footprint over
boot latency
Andrei Tatar [Thu, 14 Mar 2024 15:38:47 +0000 (16:38 +0100)]
lib/ukcpio: Remove special handling of "."
This change removes the special handling of "." on cpio extraction, as
it introduced an unnecessary strcmp on every path, prevented mode bits
from being applied on the destination root, and produced a warning at
runtime.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Approved-by: Razvan Deaconescu <razvand@unikraft.io> Reviewed-by: Sergiu Moga <sergiu@unikraft.io> Reviewed-by: Eduard Vintilă <eduard.vintila47@gmail.com>
GitHub-Closes: #1362
Andrei Tatar [Wed, 17 Apr 2024 16:13:23 +0000 (18:13 +0200)]
lib/{posix-*,ukfile}: Add ukatomic dependency
This change adds a Kconfig dependency to ukatomic on several libraries
that were written before the ukatomic split-off. This makes their using
atomic operations explicit.
Signed-off-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Robert Zamfir <georobi.016@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1393
Simon Kuenzer [Tue, 14 May 2024 16:20:05 +0000 (18:20 +0200)]
lib/ukboot: Replace NOALLOC with INITALLOC
This commit introduces the hidden configuration option
`CONFIG_LIBUKBOOT_INITALLOC` that indicates that allocator initialization
is configured with `lib/ukboot`.
We do this in order to be inline with the scheduler configuration
(`Config.uk`) and to have the possibility to add dependencies to the
corresponding `choice` of the allocator initialization configuration in
the future.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1413
Simon Kuenzer [Tue, 14 May 2024 16:19:51 +0000 (18:19 +0200)]
lib/ukboot: Replace NOSCHED with INITSCHED
This commit introduces the hidden configuration option
`CONFIG_LIBUKBOOT_INITSCHED` that indicates that scheduler initialization
is configured with `lib/ukboot`.
The previous check for `!CONFIG_LIBUKBOOT_NOSCHED` was not sufficient,
as the option is not available if `CONFIG_LIBUKBOOT_NOALLOC` is set. The
reason is the dependency expressed for the corresponding `choice` in
the libraries' `Config.uk`. This would incorrectly mean that a scheduler is
being initialized.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1413
Sergiu Moga [Wed, 15 May 2024 14:10:38 +0000 (17:10 +0300)]
uk/plat: Mark MRD validation flags variable in macro as maybe unused
There may be unused warnings for this variable in builds that do not
have assertions enabled. Therefore, make it so that it does not generate
such warnings if assertions are disabled.
Sergiu Moga [Sun, 29 Oct 2023 08:27:34 +0000 (10:27 +0200)]
uk/plat: Add `UK_ASSERT_VALID_MRD_*` to assert on invalid mrd types
Implement a memory region type specific spin-off of `UK_ASSERT` that
will throw assertions specific to the type of the memory region being
asserted
As a basis, MRDs must meet the following, generic, criteria:
- must be of only one valid type as per `UK_ASSERT_VALID_MRD_TYPE`
- must only have valid flags as per `UK_ASSERT_VALID_MRD_FLAGS`
- memory region is not empty or of length 0
- virtual/physical base addresses are page-aligned
- resource in-page offset must be in the range [0, PAGE_SIZE)
For now, two additional specialized MRD validation macros were added.
In the case of `UK_ASSERT_VALID_FREE_MRD`, they must meet the
following criteria:
- must meet the criteria of a general valid memory region descriptor
- virtual/physical base addresses are equal
- region is aligned end-to-end, therefore length is multiple of
PAGE_SIZE times region's page count and the resource's
in-page offset must be 0
In the case of `UK_ASSERT_VALID_KERNEL_MRD`, they must meet the
- must meet the criteria of a general valid memory region descriptor
- region is aligned end-to-end, therefore length is multiple of
PAGE_SIZE times region's page count and the resource's
in-page offset must be 0
That being said, with the addition of these macros, sprinkle around
some mrd validations.
After successfully fetching the `initrd` memory region, use
`UK_ASSERT_VALID_MRD to validate it against a predefined set of
criteria that generally apply to all memory regions, regardless
of type.
`ukplat_memregion_alloc` makes use of free memory regions in order
to break a chunk out of them and assign it a given type depending
on the requested allocation. Make sure that the free memory region
we use meets the criteria that all free memory regions must have
by using `UK_ASSERT_VALID_FREE_MRD` before modifying it.
Validate all memory regions that we process during the execution
of `ukplat_memregion_list_coalesce` against the general criteria
that must be met by all memory region descriptors.
When not using `HAVE_PAGING`, we go through each memory region in
the list and drop those that are outside the static boot page tables
since we can't extend it. Thus, make sure we also make sure that
these memory region descriptors are valid w.r.t. the general
criteria that must be met by all memory region descriptors.
Properly validate mrds during `ukplat_paging_init` such that:
- for the first loop that adds free memory region descriptors to the
frame allocator, we validate with `UK_ASSERT_VALID_FREE_MRD`
- for the second loop that unmaps according to `UKPLAT_MEMRF_UNMAP`
memory region descriptors, we have a one use-case, very particular
subset of conditions that must be met by this kind of memory regions
only:
- must be non-empty and aligned end-to-end, i.e. length must
be !0 and the page count times PAGE_SIZE must be equal to length
- virtual base address must be page-aligned and a valid value
- physical base address must be 0
- page offset must be 0
- for the last loop, the one that processes `UKPLAT_MEMRF_MAP`
memory region descriptors, we enforce the general rules, i.e.
use `UK_ASSERT_VALID_MRD` and, furthermore, we also ensure
that we not somehow map free memory regions.
Lastly, when enforcing `W^X` we go through memory regions and get
permissions from there. Ensure that these processed memory region
descriptors are valid w.r.t. the predefined criteria that must be
met by all memory region descriptors.
NOTE: This is an initial attempt at validating as many directly
referenced memory region descriptors as possible. It is not exhaustive
and does not guarantee validating every single possible reference.
Sergiu Moga [Sat, 28 Oct 2023 15:57:12 +0000 (18:57 +0300)]
uk/plat/memory: Introduce `pg_off` and `pg_count` memregion fields
To make memory region management easier w.r.t. alignment handling,
define two additional fields for `struct ukplat_memregion_desc`:
- `pg_off` to represent the in-page offset from where the actual
resource this memory region is dedicated to starts
- `pg_count` to represent the length of the entire, end-to-end
page-aligned, memory region in number of pages
Thus, the definition of some other fields shall then change:
- `pbase` will be the physical page-aligned base address of the
region. This means that in order to get the actual address of a
resource, one may have to make the following basic addition:
`pbase` + `pg_off`
- `vbase` same as `pbase` but for virtual base address
- `len` will now represent the length of the resource inside the
region, not the length of the region.
E.g.
For a resource with address `0x1050` and length `0x430` the
corresponding memory region descriptor will have the following
values:
- `pbase` and `vbase` equal to `0x1000` (`PAGE_ALIGN_DOWN(0x1050)`)
- `pg_off` equal to `0x50` (`0x1050 & ~PAGE_MASK`)
- `pg_count` equal to `5` (`PAGE_COUNT(0x1050 + 0x430)`)
- `len` equal to `0x430`
The other fields (`type`, `flags`, `name`) will keep their meaning.
Now with the new structure, make sure that every call site to
`ukplat_memregion_list_insert` also initializes `pg_off` and
`pg_count` accordingly.
Most importantly, deprecate the manual alignment and restoration
of memory regions during coalescing, as it is not longer needed.
The newly introduced fields guarantee that `pbase` and `pg_count`
combined will always yield end-to-end aligned memory regions.
In the case of printing, make memory map printing functionality
show two sets of address ranges for each memory region descriptor:
one for the page-aligned start and end of the memory region and one
for the real, potentially misaligned, start and end addresses of the
memory region descriptor (the actual start/end addresses of the resource
the region is meant to map).
plat/kvm: Remove initialization of PL031 from plat
Remove the initialization of PL031 from `plat/kvm/arm/setup.c`. Now
that the driver is probed by the platform bus, there is no need to be
initialized in `setup.c`.
drivers/ukbus/platform: Add pl031 to platform devices
Add platform id for pl031. Add pl031 as compatible device for the
device-tree. The changes are made so the driver can be probed through
the platform bus.
Extend the GS_BASE read/write functionality to the FS_BASE register
and move all of it to libcontext. Additionally, make the initialization
of these routines, which depends on FSGSBASE being available, registered
as a Unikraft constructor.
Signed-off-by: Sergiu Moga <sergiu@unikraft.io> Co-authored-by: Andrei Tatar <andrei@unikraft.io> Reviewed-by: Marco Schlumpp <marco@unikraft.io> Reviewed-by: Michalis Pappas <michalis@unikraft.io> Approved-by: Simon Kuenzer <simon@unikraft.io>
GitHub-Closes: #1316
lib/ukboot: Make use of the generic (page guarded) stack allocator
Allow ukboot, if configured to, to make use of the stack allocator
library for higher flexibilty and more control over the allocators used
for stacks. Specifically, initialize the scheduler with special stack
allocators and, because of this, remove the explicit `uk_vma_advise`
from uksched's thread creation and allow these allocators to
transparently do what they were configured to.
The default initialization of the cooperative scheduler now takes a
normal stack allocator for generic thread stacks whose initial
premapped length is configurable through the
CONFIG_LIBUKBOOT_ALLOCSTACK_PREMAP_ORDER configuration entry. What is
more, we keep the auxiliary stack semantics of always having it
premapped by configuring its stack allocator to fully map any stack
that is generated through it.
lib/uksched: Give scheduler init higher granularity w.r.t. allocators
Instead of using the same allocator for stack, auxiliary stack, TLS
and other structures, allow a caller of the scheduler initialization
function to pass different allocators for these if they want to.
Implement a generic stack allocation wrapper over a given Unikraft
memory allocator. The library receives as arguments an allocator and,
if libukvmem is enabled, a virtual address space and an initial size.
The library works in three ways:
1. If libukvmem is disabled, the library will make use of the allocator
received as argument to allocate the initial allocator structure and
Unikraft compliant stacks.
2. If libukvmem is enabled, the library will make use of the allocator
received as argument to allocate the initial allocator structure and
Unikraft compliant stacks. The difference from 1. is that the virtual
address space and the initial size arguments are now available. The
two arguments will hint how much of the stack to premap whenever they
are allocated first, i.e. what is the initial stack size before
on-demand paging kicks in.
3. If libukvmem is enabled AND the configuration option of the library
integrating page guards is enabled, the allocator given as argument
will only be used once when allocating the initial allocator structure,
while the virtual address space and initial size arguments will be used
to create stack VMA's, specific to libukvmem.
lib/ukvmem: Fix out of range `vma_end` found as last element of list
Consider the following case:
+------------+
|VMA to unmap|
+------------+ next VMA
+----------+ +----------+
|mapped VMA| |mapped VMA|
+----------+ +----------+
The current state of the code would simply assign `vma_end` to the
next mapped VMA without checking if the end of the VMA to unmap is
within its range. Fix this by adding this check and ensure we do
not assign `vma_end` to the next VMA if it does not meet the
requirements.
Sergiu Moga [Mon, 19 Feb 2024 09:24:59 +0000 (11:24 +0200)]
lib/ukvmem: Make stack VMA guards size configurable and end-to-end
Add end-to-end page guards to stack VMAs and make their size be
configurable: default 4 pages for top (to counter stack declared arrays)
and 1 page for bottom (to counter too many pop instructions).
Sergiu Moga [Tue, 13 Feb 2024 14:06:56 +0000 (16:06 +0200)]
{arch,lib,plat}: Align auxstack with regular stack configuration
Everything stack size or alignment related is defined in `arch/`.
Therefore, move everything auxiliary stack related there and rename
related definitions/configurations to have them aligned with the
normal stacks. Additionally, remove the no longer useful related
method `ukplat_auxsp_alloc`.
Rares Miculescu [Sat, 30 Mar 2024 10:06:40 +0000 (12:06 +0200)]
plat/kvm/x86: Update data types from implementation
`plat/kvm/x86/time.c` includes `include/uk/plat/time.h`, where data
types are updated. Update data types from `plat/kvm/x86/time.c`, so
they match with the data types from declaration.
lib/ukvmem/arch/arm: Fix `error_code` build error with correct `esr`
Commit 4d973417284a ("lib/ukvmem/arch: Do not print error message if demand paging disabled")
introduced a build error and a warning by copy-pasting the change from
x86 to Arm. This is incorrect as the trap context from Arm does not have
the `error_code` but the `esr` field instead. Fix this by using using the
corect field and correct string format.
Cezar Craciunoiu [Thu, 28 Mar 2024 10:40:31 +0000 (12:40 +0200)]
.github/workflows: Add catalog tests and depend on them
This adds a new job that will start all catalog 'core_merge' workflows
and poll once every 30s for the results. If at least one of them fails
then it will mark it as failed, otherwise all succeeding means a pass.
Test will fail if there are other ongoing catalog 'core_merge'
workflows, to protect again concurrency problems. Rerun if necessary.
After this runs and everything passes, staging is merged into stable.
This runs only on pushes to staging.
Signed-off-by: Cezar Craciunoiu <cezar.craciunoiu@gmail.com> Reviewed-by: Alexander Jung <alex@unikraft.io> Reviewed-by: Razvan Deaconescu <razvand@unikraft.io> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1363
Cezar Craciunoiu [Thu, 21 Mar 2024 15:09:14 +0000 (17:09 +0200)]
.github/workflows: Add job to merge staging to stable
If all tests pass on a push, this should move the commits from staging
to stable. This should allow for the default branch to become
'stable' afterwards.
Signed-off-by: Cezar Craciunoiu <cezar.craciunoiu@gmail.com> Reviewed-by: Alexander Jung <alex@unikraft.io> Reviewed-by: Razvan Deaconescu <razvand@unikraft.io> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1363
Compensate for the fact that some subsystems end up terminating
threads without being timerfd aware.
For example, posix-process may try to terminate all threads of a
given process without being aware that a thread may have open fd's.
In this case, a closed thread with an open timerfd results in the
timerfd still holding a reference to the released thread to terminate
it when it is closed, resulting in a double termination or memory
corruption.
Fix this by adding a destructor that, when called, uses the `priv` field
of `struct uk_thread`, initially assigned to the timerfd structure
holding this reference, to mark this reference as NULL so that double
termination is avoided.
Signed-off-by: Sergiu Moga <sergiu@unikraft.io> Approved-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Simon Kuenzer <simon@unikraft.io>
GitHub-Closes: #1375
lib/posix-timerfd: Fix thread double free on timerfd release
Whenever a timer fd is released, the thread associated with its update
function is also released through a combination of `uk_thread_terminate`
and `uk_thread_release`. However, if the released thread is different
from the current thread then `uk_thread_terminate` also calls
`uk_thread_release` on its own, leading to a double free. This is always
the case as the release function is impossible to be called from the
update function.
Therefore, fix this by deleting the explicit call to `uk_thread_release`
and rely on the one implicitly called by `uk_thread_terminate`.
Signed-off-by: Sergiu Moga <sergiu@unikraft.io> Approved-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Simon Kuenzer <simon@unikraft.io>
GitHub-Closes: #1375
Alexander Jung [Wed, 27 Mar 2024 16:39:13 +0000 (17:39 +0100)]
doc: Refactor README.md with new layout and latest info
This commit includes a large refactor to the top-level README.md
which aims to significantly improve initial understanding of
the Unikraft project, its capabilities, ways to get started,
how to get invovled and additional information about latest
cloud deployment options with KraftCloud.
Signed-off-by: Alexander Jung <alex@unikraft.io> Reviewed-by: Felipe Huici <felipe@unikraft.io> Approved-by: Felipe Huici <felipe@unikraft.io>
The header (first descriptor) of the virtio-blk request must always
come in one piece! By aligning to its size (16), we ensure that
it will be contained entirely in one page only, i.e. the
scatter-gather list will not process it as two segments, which
would result in two descriptors.
E.g. If the address ends something like 0x...ff8 then the header
will span 0x...ff8 -> 0x...008 crossing the next page and resulting
in the scatter-gather list splitting it into two segments and
thus into two descriptors, which QEMU seems to not like.
To help with this, declare the request header at the beginning
of the virtio-blk request structure and make its allocation
16-byte aligned, guaranteeing its length will never cross the page
boundary.
Sergiu Moga [Sun, 28 Jan 2024 15:52:51 +0000 (17:52 +0200)]
lib/uksglist: Move scatter-gather list save/restore to header
It may come in handy to do scatter-gather list save and restore
in places other than the scatter-gather library. Therefore, allow
others to see the corresponding definitions by placing them in
the header.
Simon Kuenzer [Mon, 4 Mar 2024 16:15:37 +0000 (17:15 +0100)]
lib/ukstreambuf: `uk_streambuf_memcpy()`: Note regarding last byte
A streambuffer that is initialized with `UK_STREAMBUF_C_TERMSHIFT` has a
special behaviour with regard to the last byte. It is intended to carry the
C-string termination symbol `\0`.
If `uk_streambuf_memcpy()` is used, the last byte is overwritten by
subsequent calls. This can be avoided by the subsequent call of
`uk_streambuf_reserve()` by shifting the end position by one byte. However,
if `UK_STREAMBUF_C_TERMSHIFT` is set, `uk_streambuf_reserve()` must be
called with 2 bytes: 1 for the shift and another to hold the termination
symbol.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Sergiu Moga <sergiu@unikraft.io> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1352
Simon Kuenzer [Mon, 4 Mar 2024 15:52:03 +0000 (16:52 +0100)]
lib/posix-environ: Deduplicate env variables during boot
This commit removes duplicate variable entries within the `environ` array.
Duplicate entries occur
1) at compile time, if the same variable has been preconfigured several
times
2) at runtime via the library parameter `env.vars`. A variable can be
declared multiple times if a value has already been set at compile time
or if a value has been set multiple times via the parameters.
Deduplication goes through each `environ` array entry and checks whether an
entry for the same variable already exists in the previous entries (lower
positions). If this is the case, the entry is written to the position of
the found entry. Since preconfigured values (compile time) are at the low
positions and, preconfigured values can be overwritten with library
parameters. The algorithm is designed not to rely on any memory allocators,
as we have to analyze and unify the entries very early during boot.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Sergiu Moga <sergiu@unikraft.io> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1352
Simon Kuenzer [Mon, 4 Mar 2024 15:51:07 +0000 (16:51 +0100)]
lib/nolibc: Add prototypes for `chdir()`, `fchdir()`
Introduces the prototypes for `chdir()`, `fchdir()` with `<unistd.h>`
if `lib/vfscore` is part of the build. `lib/vfscore` is providing
the libc wrapper implementation.
Signed-off-by: Simon Kuenzer <simon@unikraft.io> Reviewed-by: Sergiu Moga <sergiu@unikraft.io> Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com> Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1352
projects / unikraft / unikraft.git / log