Yuri Volchkov [Mon, 4 Mar 2019 17:46:28 +0000 (18:46 +0100)]
lib/vfscore: fix conflicts with newlib
Consists of multiple tiny fixes:
- syscalls.c: include dirent.h after unistd.h, because it requires
off_t, ino_t and size_t declarations
- vfs.h: kill unused OPEN_MAX definition
- vnode.c: prefix S_BLKSIZE
Wei Chen [Fri, 1 Mar 2019 14:54:38 +0000 (22:54 +0800)]
plat/kvm: arm64: Ensure there is no stall data in cache when boot
We will disable MMU and CACHE before pagetable is ready. This
means we will change memory with cache disabled, so we need to
invalidate the cache to ensure there is no stall data in it.
But we don't know the size of the RAM either. And it would be
expensive to invalidate the whole cache. In this case, just
just need to invalidate what we are going to use:
DTB, TEXT, DATA, BSS, pagetables and bootstack.
Wei Chen [Fri, 1 Mar 2019 14:54:35 +0000 (22:54 +0800)]
plat/kvm: arm64: Build L3 page table for range greater than 2MB
Current L3 table build function only supports to build L3 table for
memory range not exceeded than 2MB. But as we mentioned in previous
patch, the image size will be greater than 2MB. So in this case, build
L3 table for bigger image is not convenient. We have improve the
function of building L3 table to support bigger memory range.
Wei Chen [Fri, 1 Mar 2019 14:54:34 +0000 (22:54 +0800)]
plat/kvm: arm64: Calculate pagetable size dynamically by image size
The previous L3 page table size is a fixed value based on the
assumption that, image size will not exceed 2MB. But that's not
true, some applications like node.js will be about 40MB after
converting to Unikernel. So in this patch, we calculate the
page table size by real image size. In order to mapping the
image occupied memory to L3 page table, we round up the image
size with 2MB. If we don't round up the image size with 2MB.
Some memory that is not occupied by image but shared the same
2MB block with image tail will be unmapped in page table.
Wei Chen [Fri, 1 Mar 2019 14:54:32 +0000 (22:54 +0800)]
plat/kvm: Invalidate D-Cache that may have been modified for Arm64
DTB, TEXT and RODATA would not be modified while booting, so we just
need to invalidate and clean the area that may have been modified:
DATA, BSS, PAGETABLE and BOOTSTACK.
Wei Chen [Fri, 1 Mar 2019 14:54:30 +0000 (22:54 +0800)]
plat/kvm: Force align the stack pointer for Arm64 EL1 exceptions
If we enable the SCTLR_ELx.SA, Arm64 stack pointer must be aligned to
16-byte before being used as base address. But some valid EL1 exceptions
don't guarantee that SP_EL1 was aligned when entering the exceptions.
Wei Chen [Fri, 1 Mar 2019 14:54:27 +0000 (22:54 +0800)]
arch/include: Add a _regs member to save origial stack pointer
There is a issue about stack alignment check in url[1]. There
might be case where the stack pointer is not aligned at EL1
exception. We will force align the stack for EL1 exceptions,
so we add a sp to save original stack pointer in this patch.
Wei Chen [Fri, 1 Mar 2019 14:54:26 +0000 (22:54 +0800)]
plat/kvm: Get VIRT_BITS from id_aa64mmfr0_el1.PARange for Arm64
As we use VA == PA mapping, so the VIRT_BITS must be the same as
PA_BITS. We can get PA_BITS from ID_AA64MMFR0_EL1.PARange and the
TxSZ of TCR_INIT_FLAGS will be calculate dynamically.
Wei Chen [Fri, 1 Mar 2019 14:54:20 +0000 (22:54 +0800)]
plat: Clean up kernel image symbols
There is a potential undefined behaviour for pointer comparision.
How to avoid this gcc optimization is under discussing. Since it
is not a block issue. I will follow it up in the future.
Provide macro definitions for text,bss,data... is the precondition
for future solution.
Wei Chen [Fri, 1 Mar 2019 14:54:19 +0000 (22:54 +0800)]
plat/common: Move global variables defined in link scripts to a header
The global variables defined in link scripts like _text, _etext and _end
are declared in multiple files. We use a header to include these global
variables to avoid so many duplicated declarations.
Wei Chen [Fri, 1 Mar 2019 14:54:18 +0000 (22:54 +0800)]
plat/include: Use macro-ed helper to simplify ioreg_read/write for Arm64
As discussed in mailing list [1], we can use macro-ed helpers to avoid
having to write 4 times of the same things of ioreg_read_8/16/32/64 and
ioreg_write8/16/32/64.
Wei Chen [Fri, 1 Mar 2019 14:54:17 +0000 (22:54 +0800)]
build: Remove the check of '--enable-default-pie' in Makefile.uk
As discussed in mailing list [1], the check for '--enable-default-pie'
is unnecessary, because GCC supports '-no-pie' as soon as it.
We have checked '-no-pie' already. So we can remove the check for
'--enable-default-pie' to get rid of the console command.
Vlad Badoiu [Wed, 9 Jan 2019 13:52:44 +0000 (13:52 +0000)]
support/build: Add support for the tar.xz and .txz format
This patch adds the .txz and tar.xz formats into the build system.
This is needed for the C++ libraries since LLVM uses tar.xz
for their releases (http://releases.llvm.org/download.html).
Simon Kuenzer [Tue, 19 Feb 2019 15:50:29 +0000 (16:50 +0100)]
lib/nolibc: Provide stdbool.h
Introduce a minimal `stdbool.h` header that provides definitions
of `bool`, `true`, and `false`.
The file is copied from LLVM Clang 7.0.1 (https://clang.llvm.org/).
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Yuri Volchkov [Mon, 18 Feb 2019 14:54:40 +0000 (15:54 +0100)]
lib/vfscore: remove sys prefix from vfs api functions
Fucntions sys(mount|umount|umount2|sync) are complete function
calls. User applications should use them directly anyways. Let's
rename them accordingly.
Signed-off-by: Yuri Volchkov <yuri.volchkov@neclab.eu> Reviewed-by: Simon Kuenzer <simon.kuenzer@neclab.eu>
Yuri Volchkov [Mon, 18 Feb 2019 14:54:31 +0000 (15:54 +0100)]
lib/vfscore: introduce fget, fdrop, fdalloc
These functions are used in the imported code, which is not enabled
yet. This required to expand functionality of vfscore_install_fd and
friends with locking and reference counting.
Simon Kuenzer [Sun, 17 Feb 2019 18:46:46 +0000 (19:46 +0100)]
plat/kvm: Acknowledge all interrupts
Acknowledge interrupts even in the case when there was no
handler for it. We do this to (1) compensate potential
spurious interrupts like currently happening on virtio,
and (2) to minimize impact on device drivers that share
one interrupt line that got previously disabled. Whenever
an interrupt was unhandled a critical warning is still
printed to the kernel console.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Simon Kuenzer [Tue, 12 Feb 2019 17:07:49 +0000 (18:07 +0100)]
lib/ukdebug: Align hexdump with print system
With the recent update of the print system, the hexdump functionality
was not adopted. This made this functionality useless. With this
commit hexdump is align with the printing again system. For instance,
uk_hexdumpd() is only enabled in the code when either UK_DEBUG is
defined for the compilation unit or debug messages are enabled
globally. uk_hexdumpk() depends on having kernel console enabled and
on the configured maximum message level.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Dafna Hirschfeld [Tue, 12 Feb 2019 13:03:11 +0000 (14:03 +0100)]
plat/xen: arm: Initialize the bss section to 0
Clear the bss section during boot so that it is initialized with 0.
Additonally, we make sure with the linker script that all
bss-related symbols are placed within the bss area marked with
_end and __bss_start when the compile option "dropping of unused
symbols" is enabled.
Simon Kuenzer [Wed, 13 Feb 2019 13:07:37 +0000 (14:07 +0100)]
plat/xen: CPU definition for Arm32
Introduce minimal CPU definitons to enable compiling for Arm32.
This commits also removes x86 specific CPU functions from the
Arm32 build unit and introduces CONFIG_XN_HVMLITE as invisible
menu configuraiton. Its intention is to support switching
between PV and PVH mode later. For now, we set PV mode for x86
and PVH mode for Arm.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Simon Kuenzer [Wed, 13 Feb 2019 12:26:26 +0000 (13:26 +0100)]
plat/xen: Split emergency and hypervisor console
Splits the two console implemetations (hvc, emergency) from each other.
Similar to plat/kvm, a console option menu is populated in the platform
configuration where each console can be individually selected as output
for kernel or debug messages. Because hvc is currently not yet supported
on Arm32, we set the debug console as default output on Arm32.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Simon Kuenzer [Tue, 12 Feb 2019 16:09:49 +0000 (17:09 +0100)]
plat/xen: Grant table support as menu option
Introduces a menu option to enable or disable grant table support.
The reason is that our grant table implementation depends on
libuklock, libuksched, and libukalloc. A minimal configuration
requires these libraries selected, too. This may be an obstacle
when running implementation written in higher-level languages.
Additionally, some of these libraries are currenlty not supported
on Arm32. It is wiser to give an option for disabling grant table
support.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Simon Kuenzer [Mon, 11 Feb 2019 17:01:10 +0000 (18:01 +0100)]
build: Notice user when to do `make clean`
The little helper script `configupdate` did not detect any config changes
with `.config` anymore because it did prefix `CONFIG_` prefix for the
configuration symbols.
This commit makes sure that `configupdate` can detect changes
(e.g., CPU architecture, code optimizations) that requires `make clean`
before continuing. It is still called automatically by the build system.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Yuri Volchkov [Tue, 12 Feb 2019 22:26:54 +0000 (23:26 +0100)]
lib/ukboot: include uk/essentials.h in weak_main
In case of compiling with newlib enabled, essentials.h is not getting
included. Because of that, compiler does not know what '__weak' means,
and the compilation fails. Let's include essentials.h explicitly.
Signed-off-by: Yuri Volchkov <yuri.volchkov@neclab.eu> Reviewed-by: Simon Kuenzer <simon.kuenzer@neclab.eu>
Simon Kuenzer [Fri, 8 Feb 2019 10:38:08 +0000 (11:38 +0100)]
plat/kvm: Disconnect console options from ukdebug
Like on the other platforms, the console options for debugging
messages should not be hidden when global debug output is not
enabled. One may still want to enable individual debugging for a
particular library or file. This commit makes this configuration
option independent in the menu.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Yuri Volchkov [Fri, 4 Jan 2019 16:18:00 +0000 (17:18 +0100)]
lib/ukboot: move weak main in separate sub-lib
Link Time Optimization does not work well with incremental linking (ld
-r), which is heavily used by Unikraft.
The ukboot has a weak main() function, which is supposed to be called
if no other library/application provided a proper main. However gcc
resolves main() to whatever is available while linking the
libukboot.ld.o. Which is the weak one.
In the upcoming gcc v9 release a better support of incremental link
optimization is announced. Which hopefully will fix this issue. But
for now, let's fix it with a workaround.
Let's create a sublib under the ukboot, containing only the weak
main. This way we will leave no chance for the first "ld -r" to link
with the wrong main. The next "ld -r" will have all the objects, so it
will be able to make correct decision.
Florian Schmidt [Thu, 17 Jan 2019 12:39:29 +0000 (13:39 +0100)]
support: don't throw error message when not in a git repo
The gitsha1 script is used to create a version string for unikraft that
includes the git hash. However, if we are not building from a git repo
checkout, it will throw error messages on the command line. Suppress
these because there's little gain in them.
Signed-off-by: Florian Schmidt <florian.schmidt@neclab.eu> Reviewed-by: Simon Kuenzer <simon.kuenzer@neclab.eu>
Simon Kuenzer [Thu, 31 Jan 2019 11:12:52 +0000 (11:12 +0000)]
build: `print-libs`, `print-objs`, `print-srcs` depend on `.config`
As soon as no configuration `.config` is available, the Make targets
`print-libs`, `print-objs`, and `print-srcs` will return an error
message. Without a configuration file these targets are not able
to do any listing of which units are used or created by a build.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Simon Kuenzer [Thu, 31 Jan 2019 11:12:50 +0000 (11:12 +0000)]
build: `distclean`, `properclean` independent of `.config`
Until now, the targets `distclean` and `properclean` were not
available if `.config` file is not present. This does not make sense,
because these targets have nothing to do with the configuration. They
just clean everything (except .config in case of properclean).
Since these targets use some functions declared in
`support/build/Makefile.rules`, we include this file at an earlier
point.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Yuri Volchkov [Tue, 4 Dec 2018 17:16:26 +0000 (18:16 +0100)]
lib/nolibc/string: fix underflow in strnlen
The memchr return NULL if it did not find '\0' character. The strnlen
did not take this into account and always did subtraction to count the
string length.
Return the provided length limit if the line ending is not found
within this limit.
Simon Kuenzer [Wed, 30 Jan 2019 23:44:58 +0000 (23:44 +0000)]
lib/ukdebug: Fix spelling in configuration menu
The help text of LIBUKDEBUG_PRINTK was speaking about debug messages
although the option configures kernel messages.
The option LIBUKDEBUG_PRINTK_INFO enables all types of kernel messages
instead of debug messages.
The message redirection option does not have a dependency on
LIBUKDEBUG_PRINTD neither on LIBUKDEBUG_PRINTK.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Disabling interrupts by virtio_netdev_recv() is not needed because in
normal cases they are already disabled by the internal receive
interrupt handler, right before a uknetdev API program can call the
receive function. When the device is used in pure polling mode, queue
interrutps are never enabled.
We remove this step because of performance reasons. An assertion is
making sure that the API is used as intended.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
Simon Kuenzer [Thu, 31 Jan 2019 15:48:42 +0000 (15:48 +0000)]
lib/uknetdev: Flag-based status report on rx and tx functions
Introduce flag based status return codes on receive and transmit
functions. They are replacing the current enum-like return codes. The
flags are able to inform the API user about additional driver
states (e.g., queue underruns).
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
The idea of having a fill-up parameter on the receive function is
inconvenient because a user never knows how many receive buffers have
to be programmed before calling the receive function.
This commit introduces registering a callback when configuring a
receive queue. This callback has to be provided by the NETDEV API user
and is called by the driver whenever it programs new receive buffers to
the receive queue. In order to still provide performance, this
allocation callback function is called for a batch of receive buffers.
This new mechanism replaces the fill-up parameter on the receive function.
This commit also adopts virtio-net in order to comply with the API change.
Signed-off-by: Simon Kuenzer <simon.kuenzer@neclab.eu> Reviewed-by: Sharan Santhanam <sharan.santhanam@neclab.eu>
projects / unikraft / unikraft.git / log