--- /dev/null
+% Xen toolstack library API/ABIs
+% Ian Campbell <ian.campbell@citrix.com>
+% Draft B
+
+# Introduction
+
+The low-level `libxenctrl` library currently has an unstable API and
+ABI and some of the hypervisor interfaces which it exposes are
+similarly unstable.
+
+However several external projects use some of these interfaces (at
+least: qemu and kexec-tools), which presents problems for distros and
+other consumers. In particular the need for spurious rebuilds of those
+components against newer versions of Xen and difficulty supporting
+parallel installation of different versions of Xen (useful during
+upgrade).
+
+This document considers whether parts of `libxenctrl` can be split out
+into new libraries with more useful API and ABI guarantees.
+
+# ABI/API Compatibility Classes
+
+Compatibility opportunities:
+
+* `LAPI` -- Library API
+* `LABI` -- Library ABI
+* `HABI` -- Hypervisor ABI (includes ioctls).
+
+Each can either be Stable (`S`), Unstable (`U`) or don't care (`x`,
+because made moot by higher level interface class, e.g. no real point
+from an application PoV to a Stable HABI behind a unstable LABI).
+
+Stable vs. Unstable is across major hypervisor version bump, always
+aim to be stable across point releases.
+
+For libraries Stable means `SONAME` major component, but forward
+compatible only (i.e. old app on new library works, new app on old
+library may not link due to e.g. new symbols. This is the normal
+SONAME expectation). XXX find a link to the sort of scheme I mean.
+
+`HABI` may include ioctls used to access those ABIs, typically these
+are already required to be stable by the relevant OS maintainers.
+
+A library interface may fall into one of these categories (I expect
+there are others and we may not want any library to use some of even
+these):
+
+* Unstable `LAPI` (`Uxx`)
+ * The Wild West
+ * Current Examples: `libxenctrl`
+
+* Stable `LAPI`, Unstable `LABI` (`SUx`)
+ * Requires application rebuild for a new Xen version, but not
+ application source code changes.
+ * Current Examples: `libxenlight`
+
+* Stable `LAPI`, Stable `LABI`, Unstable `HABI` (`SSU`)
+ * Library can be switched out via dynamic linking across
+ hypervisor upgrade (mechanism TBD, pos. distro specific, e.g.
+ symlink switched on boot). Requires application/daemon restart
+ but not rebuild (but changing hypervisor version involves a
+ reboot anyway).
+ * Current Examples: None??
+
+* Stable `LAPI`, Stable `LABI`, Stable `HABI` (`SSS`)
+ * Applications linked again a library will function against any
+ hypervisor version.
+ * Current Examples: None??
+
+# Goal
+
+Provide `SSU` or `SSS` interfaces for major external consumers of
+current libxenctrl functionality.
+
+Out of scope (for now): `SSU` or `SSS` interfaces for consumers of
+`libxenlight`. Rationale: Lets focus on fixing external consumers of
+libxenctrl first.
+
+# Known External Consumers of `libxenctrl`
+
+* qemu
+* kexec tools
+* in guest tools e.g. users of `libxenstore`, `libvchan`, and by
+ extension `grant table` and `event channel` functionality. NB:
+ `libxenstore` is already `SSU` or `SSS` (XXX?)
+
+# `libxenguest`
+
+Uses some of the same sources as `libxenctrl`. Could do with some
+rationalisation of what lives where.
+
+Not a candidate for API/ABI stability at this point.
+
+# Stable libraries
+
+These libraries will provide a stable API to all consumers going
+forward. They can be mixed and matched with Xen version.
+
+## `libxentoollog`: logging
+
+The following do not depend on any underlying Xen functionality.
+
+Interface Known external users
+----------------------------- --------------------
+`xtl_createlogger_stdiostream`
+`xtl_level_to_string`
+`xtl_log`
+`xtl_logger_destroy`
+`xtl_logv`
+`xtl_progress`
+`xtl_stdiostream_adjust_flags`
+`xtl_stdiostream_set_minlevel`
+
+## `libxenevtchn`: event channels
+
+Interacting with Event Channels via `/dev/xen/evtchn`.
+
+Interface Underlying interface Known external users
+----------------------------- -------------------------------- --------------------
+`xenevtchn_bind_interdomain` `IOCTL_EVTCHN_BIND_INTERDOMAIN` qemu-pv
+`xenevtchn_bind_unbound_port` `IOCTL_EVTCHN_BIND_UNBOUND_PORT`
+`xenevtchn_bind_virq` `IOCTL_EVTCHN_BIND_VIRQ`
+`xenevtchn_close` `close(2)` qemu-dm
+`xenevtchn_fd`
+`xenevtchn_notify` `IOCTL_EVTCHN_NOTIFY` qemu-dm, qemu-pv
+`xenevtchn_open` `open(2)` qemu-dm, qemu-pv
+`xenevtchn_pending` `read(2)` qemu-dm, qemu-pv
+`xenevtchn_unbind` `IOCTL_EVTCHN_UNBIND` qemu-pv
+`xenevtchn_unmask` `write(2)` qemu-dm, qemu-pv
+
+## `libxengnttab`: grant tables
+
+Interacting with Grant Tables via `/dev/xen/gnt{shr,alloc}`.
+
+Interface Underlying interface Known external users
+--------------------------------- ------------------------------ --------------------
+`xengnttab_map_domain_grant_refs` `IOCTL_GNTDEV_SET_MAX_GRANTS`
+`xengnttab_map_grant_ref_notify` `IOCTL_GNTDEV_MAP_GRANT_REF`
+`xengnttab_map_grant_ref` `IOCTL_GNTDEV_MAP_GRANT_REF` qemu-pv
+`xengnttab_map_grant_refs` `IOCTL_GNTDEV_SET_MAX_GRANTS` qemu-pv
+`xengnttab_unmap` `munmap(2)` qemu-pv
+`xengnttab_open` `open(2)` qemu-pv
+`xengnttab_close` `close(2)` qemu-pv
+`xengnttab_set_max_grants` `IOCTL_GNTDEV_SET_MAX_GRANTS`
+`xengntshr_close` `close(2)`
+`xengntshr_unshare` `munmap(2)`
+`xengntshr_open` `open(2)`
+`xengntshr_share_page_notify` `IOCTL_GNTALLOC_ALLOC_GREF`
+`xengntshr_share_pages` `IOCTL_GNTALLOC_ALLOC_GREF`
+
+`xengnt???_*_notify` also use `IOCTL_GNTALLOC_SET_UNMAP_NOTIFY`,
+`IOCTL_GNTALLOC_SET_UNMAP_NOTIFY`, etc
+
+## `libxencall`: making hypercalls
+
+Making hypercalls and allocating suitable memory to use as hypercall
+pointer ("GUEST_HANDLE") arguments.
+
+Interface Underlying interface Known external users
+--------------------------------- ------------------------------ --------------------
+`xencall_open` `open(2)`
+`xencall_close` `close(2)`
+`xencall0`, ..., `xencall5` `IOCTL_PRIVCMD_HYPERCALL`
+`xencall_alloc_buffer` `xencall_alloc_buffer_pages`
+`xencall_alloc_buffer_pages` `mmap(2)`, `madvise(2)`
+`xencall_free_buffer` `xencall_free_buffer_pages`
+`xencall_free_buffer_pages` `munmap(2)`, `madvise(2)`
+
+## `libxenforeignmemory`: privileged foreign mappings
+
+Mapping foreign guest memory.
+
+Interface Underlying interface Known external users
+--------------------------------- ------------------------------ --------------------
+`xenforeignmemory_open` `open(2)`
+`xenforeignmemory_close` `close(2)`
+`xenforeignmemory_map` `IOCTL_PRIVCMD_MMAPBATCH` etc
+
+# Unstable libraries
+
+These libraries do not provide a stable interface and are required to
+be updated in lockstep with the hypervisor.
+
+## `libxenctrl`
+
+Gathered by:
+
+ $ objdump -T tools/libxc/libxenctrl.so| awk '$4 ~ /\.text/ { print $7 }' | sort -u
+
+The following proposes some functional groupings via some proposed
+split library names. In some cases we may also wish to consider
+replacing an API with one which can be properly maintained going
+forwards. e.g.:
+
+- perhaps replacing domctl's used by qemu with new stable
+ hypercall ABIs and reflecting that in new library APIs.
+- perhaps exposing more constrained versions of some broad interfaces
+ for external users.
+
+In addition to these there are many inlines in `xenctrl.h`. Where
+possible these should become either private functions or regular
+functions exported by the library.
+
+Notes:
+
+* Things marked "private" are generally defined in `xc_private.h` or
+ are otherwise supposedly internal but leaked.
+* Things marked "external, via ..." are generally expected to be used
+ via `...` (often a macro).
+* There are lots of static functions in the `xc_*` namespace.
+
+### Logging
+
+The following are already logically somewhat separate. They do not
+depend on any underlying Xen functionality.
+
+Interface Known external users
+----------------------------- --------------------
+`xtl_createlogger_stdiostream`
+`xtl_level_to_string`
+`xtl_log`
+`xtl_logger_destroy`
+`xtl_logv`
+`xtl_progress`
+`xtl_stdiostream_adjust_flags`
+`xtl_stdiostream_set_minlevel`
+
+### Basic interface
+
+ - xc_interface_close
+ - xc_interface_open
+
+### Event channels
+
+Interacting with Event Channels via `__HYPERVISOR_event_channel_op`
+(`EVTCHNOP_*`).
+
+Interface Underlying interface Known external users
+----------------------------- -------------------------------- --------------------
+`xc_evtchn_alloc_unbound` `EVTCHNOP_alloc_unbound` qemu PV dombuilder
+`xc_evtchn_reset` `EVTCHNOP_reset`
+`xc_evtchn_status` `EVTCHNOP_status`
+
+### Grant tables
+
+Interacting with Grant Tables via `__HYPERVISOR_grant_table_op`
+(`GNTTABOP_*`)`.
+
+Interface Underlying interface Known external users
+--------------------------------- ------------------------------ --------------------
+`xc_gnttab_op` `__HYPERVISOR_grant_table_op`
+`xc_gnttab_get_version` `GNTTABOP_get_version`
+`xc_gnttab_map_table_v1` `GNTTABOP_setup_table`
+`xc_gnttab_map_table_v2` `GNTTABOP_setup_table`
+
+`xc_*_notify` also use `IOCTL_GNTALLOC_SET_UNMAP_NOTIFY`,
+`IOCTL_GNTALLOC_SET_UNMAP_NOTIFY`, etc
+
+### Hypercalls
+
+#### Wrappers
+
+Interface Known in-tree users Known external users
+--------------------------------- ----------------------------- --------------------
+`do_memory_op` private private
+`xc_domctl` Save/Restore None
+`xc_sysctl` xencov, xenmon None
+
+#### Hypercall buffers
+
+Allocate and manange (and attempt to enforce the use of) memory
+suitable for passing to a hypercall as a GUEST_HANDLE argument.
+
+Interface Known in-tree users Known external users
+----------------------------------- ------------------------- --------------------
+`xc_hypercall_buffer_array_create` kexec-tools
+`xc_hypercall_buffer_array_destroy` kexec-tools
+
+The following are external via a corresponding `xc_` function:
+
+Interface Known in-tree users Known external users
+----------------------------------- ---------------------------- --------------------
+`xc__hypercall_buffer_alloc` xencov, xenlockprof, xenperf
+`xc__hypercall_buffer_alloc_pages`
+`xc__hypercall_buffer_array_alloc` kexec-tools
+`xc__hypercall_buffer_array_get`
+`xc__hypercall_buffer_free` xencov, xenlockprof, xenperf
+`xc__hypercall_buffer_free_pages`
+
+The following are private:
+
+- `xc__hypercall_bounce_post`
+- `xc__hypercall_bounce_pre`
+- `xc__hypercall_buffer_cache_release`
+
+### Mapping foreign memory
+
+All implemented in terms of `xenforeignmemory_map`.
+
+Interface Known external users
+--------------------------------- --------------------
+`xc_map_foreign_pages` qemu-pv
+`xc_map_foreign_range` qemu-dm, qemu-pv
+`xc_map_foreign_ranges`
+`xc_map_foreign_bulk` qemu-dm
+
+### Managing guest memory map
+
+Interface Underlying interface Known external users
+--------------------------------- ------------------------------ --------------------
+`xc_domain_add_to_physmap` `XENMEM_add_to_physmap` qemu-dm
+`xc_domain_populate_physmap` `XENMEM_populate_physmap`
+`xc_domain_populate_physmap_exact` `XENMEM_populate_physmap` qemu-dm
+
+### Kexec
+
+Functions specifically for use by kexec tools
+
+Interface Underlying interface
+----------------------------- ------------------------------
+`xc_kexec_exec` `KEXEC_CMD_kexec`
+`xc_kexec_get_range` `KEXEC_CMD_kexec_get_range`
+`xc_kexec_load` `KEXEC_CMD_kexec_unload`
+`xc_kexec_unload` `KEXEC_CMD_kexec_unload`
+
+### Unclassified remains
+
+_Note_: Not everything which belongs in one of the above categorisations has
+ actually been moved to a suitable home.
+
+Symbols:
+
+ - bitmap_64_to_byte
+ - bitmap_byte_to_64
+ - discard_file_cache
+ - read_exact
+ - write_exact
+ - writev_exact
+ - xc_add_mmu_update
+ - xc_alloc_mmu_updates
+ - xc_assign_device
+ - xc_assign_dt_device
+ - xc_availheap
+ - xc_clear_domain_pages
+ - xc_clear_last_error
+ - xc_copy_to_domain_page
+ - xc_core_arch_auto_translated_physmap
+ - xc_core_arch_get_scratch_gpfn
+ - xc_core_arch_gpfn_may_present
+ - xc_core_arch_map_p2m
+ - xc_core_arch_map_p2m_writable
+ - xc_core_arch_memory_map_get
+ - xc_core_shdr_get
+ - xc_core_shdr_set
+ - xc_cpumap_alloc
+ - xc_cpumap_clearcpu
+ - xc_cpumap_setcpu
+ - xc_cpumap_testcpu
+ - xc_cpu_offline
+ - xc_cpu_online
+ - xc_cpupool_addcpu
+ - xc_cpupool_create
+ - xc_cpupool_destroy
+ - xc_cpupool_freeinfo
+ - xc_cpupool_getinfo
+ - xc_cpupool_infofree
+ - xc_cpupool_movedomain
+ - xc_cpupool_removecpu
+ - xc_cputopoinfo
+ - xc_deassign_device
+ - xc_deassign_dt_device
+ - xc_disable_turbo
+ - xc_domain_bind_pt_irq
+ - xc_domain_bind_pt_isa_irq
+ - xc_domain_bind_pt_pci_irq
+ - xc_domain_bind_pt_spi_irq
+ - xc_domain_cacheflush
+ - xc_domain_claim_pages
+ - xc_domain_create
+ - xc_domain_create_config
+ - xc_domain_debug_control
+ - xc_domain_decrease_reservation
+ - xc_domain_decrease_reservation_exact
+ - xc_domain_destroy
+ - xc_domain_disable_migrate
+ - xc_domain_dumpcore
+ - xc_domain_dumpcore_via_callback
+ - xc_domain_get_cpu_usage
+ - xc_domain_get_guest_width
+ - xc_domain_getinfo
+ - xc_domain_getinfolist (`XEN_SYSCTL_getdomaininfolist`). Used by qemu-builder
+ - xc_domain_get_machine_address_size
+ - xc_domain_get_pod_target
+ - xc_domain_get_tsc_info
+ - xc_domain_hvm_getcontext
+ - xc_domain_hvm_getcontext_partial
+ - xc_domain_hvm_setcontext
+ - xc_domain_increase_reservation
+ - xc_domain_increase_reservation_exact
+ - xc_domain_iomem_permission
+ - xc_domain_ioport_mapping
+ - xc_domain_ioport_permission
+ - xc_domain_irq_permission
+ - xc_domain_maximum_gpfn
+ - xc_domain_max_vcpus
+ - xc_domain_memory_exchange_pages
+ - xc_domain_memory_mapping
+ - xc_domain_node_getaffinity
+ - xc_domain_node_setaffinity
+ - xc_domain_nr_gpfns
+ - xc_domain_p2m_audit
+ - xc_domain_pause
+ - xc_domain_pin_memory_cacheattr (`XEN_DOMCTL_pin_mem_cacheattr`). Used by qemu-dm.
+ - xc_domain_resume
+ - xc_domain_send_trigger
+ - xc_domain_set_access_required
+ - xc_domain_setdebugging
+ - xc_domain_sethandle
+ - xc_domain_set_machine_address_size
+ - xc_domain_set_max_evtchn
+ - xc_domain_setmaxmem (`XEN_DOMCTL_max_mem`). Used by qemu-dm.
+ - xc_domain_set_memmap_limit
+ - xc_domain_set_memory_map
+ - xc_domain_set_pod_target
+ - xc_domain_set_target
+ - xc_domain_set_time_offset
+ - xc_domain_set_tsc_info
+ - xc_domain_set_virq_handler
+ - xc_domain_setvnuma
+ - xc_domain_shutdown (`SCHEDOP_remote_shutd`). Used by qemu-dm.
+ - xc_domain_subscribe_for_suspend
+ - xc_domain_suppress_spurious_page_faults
+ - xc_domain_unbind_msi_irq
+ - xc_domain_unbind_pt_irq
+ - xc_domain_unbind_pt_spi_irq
+ - xc_domain_unpause
+ - xc_domain_update_msi_irq
+ - xc_enable_turbo
+ - xc_error_code_to_desc
+ - xc_ffs16
+ - xc_ffs32
+ - xc_ffs64
+ - xc_ffs8
+ - xc_flask_access
+ - xc_flask_add_device
+ - xc_flask_add_iomem
+ - xc_flask_add_ioport
+ - xc_flask_add_pirq
+ - xc_flask_avc_cachestats
+ - xc_flask_avc_hashstats
+ - xc_flask_context_to_sid
+ - xc_flask_del_device
+ - xc_flask_del_iomem
+ - xc_flask_del_ioport
+ - xc_flask_del_pirq
+ - xc_flask_getavc_threshold
+ - xc_flask_getbool_byid
+ - xc_flask_getbool_byname
+ - xc_flask_getenforce
+ - xc_flask_load
+ - xc_flask_op
+ - xc_flask_policyvers
+ - xc_flask_relabel_domain
+ - xc_flask_setavc_threshold
+ - xc_flask_setbool
+ - xc_flask_setenforce
+ - xc_flask_sid_to_context
+ - xc_flush_mmu_updates
+ - xc_get_cpufreq_avgfreq
+ - xc_get_cpufreq_para
+ - xc_get_cpuidle_max_cstate
+ - xc_getcpuinfo
+ - xc_get_cpumap_size
+ - xc_get_device_group
+ - xc_get_hvm_param (Legacy, replaced by `xc_hvm_param_get`)
+ - xc_get_last_error
+ - xc_get_machine_memory_map. Used by kexec-tools
+ - xc_get_max_cpus. Used by kexec-tools
+ - xc_get_max_nodes
+ - xc_get_mem_access
+ - xc_get_nodemap_size
+ - xc_get_online_cpus
+ - xc_get_pfn_list
+ - xc_get_pfn_type_batch
+ - xc_get_tot_pages
+ - xc_get_vcpu_migration_delay
+ - xc_hvm_create_ioreq_server
+ - xc_hvm_destroy_ioreq_server
+ - xc_hvm_get_ioreq_server_info
+ - xc_hvm_inject_msi (`HVMOP_inject_msi`). Used by qemu-dm.
+ - xc_hvm_inject_trap
+ - xc_hvm_map_io_range_to_ioreq_server
+ - xc_hvm_map_pcidev_to_ioreq_server
+ - xc_hvm_modified_memory (`HVMOP_modified_memory`). Used by qemu-dm.
+ - xc_hvm_param_get (`HVMOP_get_param`)
+ - xc_hvm_param_set (`HVMOP_set_param`)
+ - xc_hvm_set_ioreq_server_state
+ - xc_hvm_set_isa_irq_level (`HVMOP_set_isa_irq_level`). Used by qemu-dm.
+ - xc_hvm_set_mem_type (`HVMOP_set_mem_type`). Used by qemu-dm.
+ - xc_hvm_set_pci_intx_level (`HVMOP_set_pci_intx_level`). Used by qemu-dm.
+ - xc_hvm_set_pci_link_route (`HVMOP_set_pci_link_route`). Used by qemu-dm.
+ - xc_hvm_track_dirty_vram (`HVMOP_track_dirty_vram`). Used by qemu-dm.
+ - xc_hvm_unmap_io_range_from_ioreq_server
+ - xc_hvm_unmap_pcidev_from_ioreq_server
+ - xc_lockprof_query
+ - xc_lockprof_query_number
+ - xc_lockprof_reset
+ - xc_machphys_mfn_list
+ - xc_make_page_below_4G
+ - xc_map_domain_meminfo
+ - xc_maximum_ram_page
+ - xc_mca_op
+ - xc_mem_access_disable_emulate
+ - xc_mem_access_enable_emulate
+ - xc_memalign
+ - xc_mem_paging_disable
+ - xc_mem_paging_enable
+ - xc_mem_paging_evict
+ - xc_mem_paging_load
+ - xc_mem_paging_nominate
+ - xc_mem_paging_prep
+ - xc_mem_paging_resume
+ - xc_memshr_add_to_physmap
+ - xc_memshr_audit
+ - xc_memshr_control
+ - xc_memshr_debug_gfn
+ - xc_memshr_debug_gref
+ - xc_memshr_domain_resume
+ - xc_memshr_nominate_gfn
+ - xc_memshr_nominate_gref
+ - xc_memshr_ring_disable
+ - xc_memshr_ring_enable
+ - xc_memshr_share_gfns
+ - xc_memshr_share_grefs
+ - xc_mmuext_op
+ - xc_monitor_disable
+ - xc_monitor_enable
+ - xc_monitor_mov_to_msr
+ - xc_monitor_resume
+ - xc_monitor_singlestep
+ - xc_monitor_software_breakpoint
+ - xc_monitor_write_ctrlreg
+ - xc_nodemap_alloc
+ - xc_numainfo
+ - xc_pcitopoinfo
+ - xc_perfc_query
+ - xc_perfc_query_number
+ - xc_perfc_reset
+ - xc_physdev_map_pirq
+ - xc_physdev_map_pirq_msi
+ - xc_physdev_pci_access_modify
+ - xc_physdev_unmap_pirq
+ - xc_physinfo
+ - xc_pm_get_cxstat
+ - xc_pm_get_max_cx
+ - xc_pm_get_max_px
+ - xc_pm_get_pxstat
+ - xc_pm_reset_cxstat
+ - xc_pm_reset_pxstat
+ - xc_psr_cmt_attach
+ - xc_psr_cmt_detach
+ - xc_psr_cmt_enabled
+ - xc_psr_cmt_get_data
+ - xc_psr_cmt_get_domain_rmid
+ - xc_psr_cmt_get_l3_cache_size
+ - xc_psr_cmt_get_l3_event_mask
+ - xc_psr_cmt_get_l3_upscaling_factor
+ - xc_psr_cmt_get_total_rmid
+ - xc_readconsolering
+ - xc_report
+ - xc_report_error
+ - xc_report_progress_single
+ - xc_report_progress_step
+ - xc_reportv
+ - xc_resource_op
+ - xc_sched_arinc653_schedule_get
+ - xc_sched_arinc653_schedule_set
+ - xc_sched_credit2_domain_get
+ - xc_sched_credit2_domain_set
+ - xc_sched_credit_domain_get
+ - xc_sched_credit_domain_set
+ - xc_sched_credit_params_get
+ - xc_sched_credit_params_set
+ - xc_sched_id
+ - xc_sched_rtds_domain_get
+ - xc_sched_rtds_domain_set
+ - xc_sedf_domain_get
+ - xc_sedf_domain_set
+ - xc_send_debug_keys
+ - xc_set_broken_page_p2m
+ - xc_set_cpufreq_gov
+ - xc_set_cpufreq_para
+ - xc_set_cpuidle_max_cstate
+ - xc_set_hvm_param (`HVMOP_set_param`, legacy, replaced by `xc_hvm_param_set`). Used by qemu-dm.
+ - xc_set_mem_access
+ - xc_set_progress_prefix
+ - xc_set_sched_opt_smt
+ - xc_set_vcpu_migration_delay
+ - xc_shadow_control
+ - xc_sharing_freed_pages
+ - xc_sharing_used_frames
+ - xc_strerror
+ - xc_tbuf_disable
+ - xc_tbuf_enable
+ - xc_tbuf_get_size
+ - xc_tbuf_set_cpu_mask
+ - xc_tbuf_set_evt_mask
+ - xc_tbuf_set_size
+ - xc_test_assign_device
+ - xc_test_assign_dt_device
+ - xc_tmem_auth
+ - xc_tmem_control
+ - xc_tmem_control_oid
+ - xc_tmem_restore
+ - xc_tmem_restore_extra
+ - xc_tmem_save
+ - xc_tmem_save_done
+ - xc_tmem_save_extra
+ - xc_translate_foreign_address
+ - xc_unmap_domain_meminfo
+ - xc_vcpu_getaffinity
+ - xc_vcpu_getcontext
+ - xc_vcpu_getinfo
+ - xc_vcpu_setaffinity
+ - xc_vcpu_setcontext
+ - xc_version. Used by kexec-tools
+ - xc_vm_event_control
+ - xc_vm_event_enable
+ - xc_watchdog
+
+## `libxenguest`
+
+Gathered by:
+
+
+ $ objdump -T tools/libxc/libxenguest.so| awk '$4 ~ /\.text/ { print $7 }' | sort -u
+
+ - arch_setup_bootearly
+ - arch_setup_bootlate
+ - arch_setup_meminit
+ - cr3_to_mfn
+ - csum_page
+ - dhdr_type_to_str
+ - dump_bad_pseudophysmap_entry
+ - elf_access_ok
+ - elf_access_unsigned
+ - elf_call_log_callback
+ - elf_check_broken
+ - elf_get_ptr
+ - elf_init
+ - elf_is_elfbinary
+ - elf_load_binary
+ - elf_lookup_addr
+ - elf_mark_broken
+ - elf_memcpy_safe
+ - elf_memset_safe
+ - elf_note_desc
+ - elf_note_name
+ - elf_note_next
+ - elf_note_numeric
+ - elf_note_numeric_array
+ - elf_parse_binary
+ - elf_parse_bsdsyms
+ - elf_phdr_by_index
+ - elf_phdr_count
+ - elf_phdr_is_loadable
+ - elf_round_up
+ - elf_section_end
+ - elf_section_name
+ - elf_section_start
+ - elf_segment_end
+ - elf_segment_start
+ - elf_set_log
+ - elf_shdr_by_index
+ - elf_shdr_by_name
+ - elf_shdr_count
+ - elf_strfmt
+ - elf_strval
+ - elf_sym_by_index
+ - elf_sym_by_name
+ - elf_xen_parse
+ - elf_xen_parse_features
+ - elf_xen_parse_guest_info
+ - elf_xen_parse_note
+ - handle_tsc_info
+ - lz4_decompress_unknownoutputsize
+ - mfn_in_pseudophysmap
+ - mfn_to_cr3
+ - mfn_to_pfn
+ - pin_table
+ - populate_pfns
+ - rec_type_to_str
+ - write_split_record
+ - write_tsc_info
+ - x86_pv_domain_info
+ - x86_pv_map_m2p
+ - xc_await_suspend
+ - xc_compression_add_page
+ - xc_compression_compress_pages
+ - xc_compression_create_context
+ - xc_compression_free_context
+ - xc_compression_reset_pagebuf
+ - xc_compression_uncompress_page
+ - xc_cpuid_apply_policy
+ - xc_cpuid_check
+ - xc_cpuid_set
+ - xc_cpuid_to_str
+ - xc_domain_get_native_protocol
+ - xc_domain_restore
+ - xc_domain_save
+ - xc_dom_allocate
+ - xc_dom_alloc_page
+ - xc_dom_alloc_segment
+ - xc_dom_boot_domU_map
+ - xc_dom_boot_image
+ - xc_dom_boot_mem_init
+ - xc_dom_boot_xen_init
+ - xc_dom_build_image
+ - xc_dom_check_gzip
+ - xc_dom_compat_check
+ - xc_dom_devicetree_file
+ - xc_dom_devicetree_max_size
+ - xc_dom_devicetree_mem
+ - xc_dom_do_gunzip
+ - xc_dom_feature_translated
+ - xc_dom_find_arch_hooks
+ - xc_dom_gnttab_hvm_seed
+ - xc_dom_gnttab_init
+ - xc_dom_gnttab_seed
+ - xc_dom_kernel_check_size
+ - xc_dom_kernel_file
+ - xc_dom_kernel_max_size
+ - xc_dom_kernel_mem
+ - xc_dom_linux_build
+ - xc_dom_loginit
+ - xc_dom_log_memory_footprint
+ - xc_dom_malloc
+ - xc_dom_malloc_filemap
+ - xc_dom_malloc_page_aligned
+ - xc_dom_mem_init
+ - xc_dom_panic_func
+ - xc_dom_parse_image
+ - xc_dom_pfn_to_ptr
+ - xc_dom_pfn_to_ptr_retcount
+ - xc_dom_printf
+ - xc_dom_rambase_init
+ - xc_dom_ramdisk_check_size
+ - xc_dom_ramdisk_file
+ - xc_dom_ramdisk_max_size
+ - xc_dom_ramdisk_mem
+ - xc_dom_register_arch_hooks
+ - xc_dom_register_external
+ - xc_dom_register_loader
+ - xc_dom_release
+ - xc_dom_strdup
+ - xc_dom_try_gunzip
+ - xc_dom_unmap_all
+ - xc_dom_unmap_one
+ - xc_dom_update_guest_p2m
+ - xc_elf_set_logfile
+ - xc_exchange_page
+ - xc_get_bit_size
+ - xc_hvm_build
+ - xc_hvm_build_target_mem
+ - xc_inflate_buffer
+ - xc_linux_build
+ - xc_linux_build_mem
+ - xc_map_m2p
+ - xc_mark_page_offline
+ - xc_mark_page_online
+ - xc_query_page_offline_status
+ - xc_read_image
+ - xc_try_lz4_decode
+
+Use privcmd and evtchn device handles:
+
+ - xc_suspend_evtchn_init_exclusive. Used by xen-hptool and libxl.
+ - xc_suspend_evtchn_init_sane. Unused
+ - xc_suspend_evtchn_release. Used by xen-hptool and libxl.
+
+# Consumers of `libxenctrl` functionality
+
+## `qemu`
+
+ $ nm tools/qemu-xen-dir-remote/i386-softmmu/qemu-system-i386 | grep \\bU.xc_
+
+ - xc_domain_add_to_physmap
+ - xc_domain_bind_pt_pci_irq
+ - xc_domain_create
+ - xc_domain_destroy
+ - xc_domain_getinfo
+ - xc_domain_ioport_mapping
+ - xc_domain_max_vcpus
+ - xc_domain_memory_mapping
+ - xc_domain_pin_memory_cacheattr
+ - xc_domain_populate_physmap_exact
+ - xc_domain_setmaxmem
+ - xc_domain_shutdown
+ - xc_domain_unbind_msi_irq
+ - xc_domain_unbind_pt_irq
+ - xc_domain_unpause
+ - xc_domain_update_msi_irq
+ - xc_evtchn_alloc_unbound
+ - xenevtchn_bind_interdomain
+ - xenevtchn_close
+ - xenevtchn_fd
+ - xenevtchn_notify
+ - xenevtchn_open
+ - xenevtchn_pending
+ - xenevtchn_unbind
+ - xenevtchn_unmask
+ - xc_get_hvm_param
+ - xc_hvm_inject_msi
+ - xc_hvm_modified_memory
+ - xc_hvm_set_isa_irq_level
+ - xc_hvm_set_mem_type
+ - xc_hvm_set_pci_intx_level
+ - xc_hvm_set_pci_link_route
+ - xc_hvm_track_dirty_vram
+ - xc_interface_close
+ - xc_interface_open
+ - xc_linux_build
+ - xenforeignmemory_map
+ - xenforeignmemory_open
+ - xenforeignmemory_unmap
+ - xc_physdev_map_pirq
+ - xc_physdev_map_pirq_msi
+ - xc_physdev_unmap_pirq
+ - xc_set_hvm_param
+
+### By functional area
+
+As a reference, this is a split of the xc_ calls made by QEMU, based on
+functionality areas
+
+"S" column is whether the interface to the left is a stable ABI or not.
+
+#### Device Model
+
+Interface S Underlying Interface S Other users
+---------------------------------- - ------------------------------ - ---------------
+`xenevtchn_bind_interdomain` Y
+`xenevtchn_close` Y
+`xenevtchn_fd` Y
+`xenevtchn_notify` Y
+`xenevtchn_open` Y
+`xenevtchn_pending` Y
+`xenevtchn_unmask` Y
+`xenforeignmemory_map` Y
+`xenforeignmemory_open` Y
+`xenforeignmemory_unmap` Y
+`xc_domain_add_to_physmap` N `XENMEM_add_to_physmap` Y libxc (dombuilder)
+`xc_domain_populate_physmap_exact` N `XENMEM_populate_physmap` Y libxc (several)
+`xc_domain_pin_memory_cacheattr` N `XEN_DOMCTL_pin_mem_cacheattr` N None
+`xc_domain_shutdown` N `SCHEDOP_remote_shutdown` Y libxl
+`xc_set_hvm_param` N `HVM_PARAM_ACPI_S_STATE` Y None
+`xc_hvm_inject_msi` N `HVMOP_inject_msi` Y None
+`xc_hvm_modified_memory` N `HVMOP_modified_memory` Y None
+`xc_hvm_set_isa_irq_level` N `HVMOP_set_isa_irq_level` Y None
+`xc_hvm_set_mem_type` N `HVMOP_set_mem_type` Y None
+`xc_hvm_set_pci_intx_level` N `HVMOP_set_pci_intx_level` Y None
+`xc_hvm_set_pci_link_route` N `HVMOP_set_pci_link_route` Y None
+`xc_hvm_track_dirty_vram` N `HVMOP_track_dirty_vram` Y None
+`xc_hvm_unmap_io_range_from_ir...` N `HVMOP_IO_RANGE_(PORT|MEM...)` Y None
+`xc_hvm_unmap_pcidev_from_iore...` N `HVMOP_unmap_io_range_from...` Y None
+`xc_interface_close`
+`xc_interface_open`
+
+#### PCI Passthrough
+
+Interface S Underlying Interface S Other users
+---------------------------------- - ------------------------------ - ---------------
+`xc_domain_bind_pt_pci_irq` N `XEN_DOMCTL_bind_pt_irq` N None
+`xc_domain_ioport_mapping` N `XEN_DOMCTL_ioport_mapping` N None
+`xc_domain_memory_mapping` N `XEN_DOMCTL_memory_mapping` N libxl
+`xc_domain_unbind_msi_irq` N `XEN_DOMCTL_unbind_pt_irq` N None
+`xc_domain_unbind_pt_irq` N `XEN_DOMCTL_unbind_pt_irq` N None
+`xc_domain_update_msi_irq` N `XEN_DOMCTL_bind_pt_irq` N None
+`xc_physdev_map_pirq` N `PHYSDEVOP_map_pirq` Y libxl
+`xc_physdev_map_pirq_msi` N `PHYSDEVOP_map_pirq` Y None
+`xc_physdev_unmap_pirq` N `PHYSDEVOP_unmap_pirq` Y libxl
+
+NB: More might be used by libxl in the future e.g. for ARM or PVH passthrough?
+
+#### PV backends
+
+All stable interfaces:
+
+ * xenevtchn_bind_interdomain (`IOCTL_EVTCHN_BIND_INTERDOMAIN`)
+ * xenevtchn_notify (`IOCTL_EVTCHN_NOTIFY`)
+ * xenevtchn_open (`open(2)`)
+ * xenevtchn_pending (`read(2)`)
+ * xenevtchn_unbind
+ * xenevtchn_unmask (`write(2)`)
+ * xengnttab_close
+ * xengnttab_map_grant_ref
+ * xengnttab_munmap
+ * xengnttab_open
+ * xengnttab_set_max_grants
+ * xenforeignmemory_map
+ * xenforeignmemory_open
+ * xenforeignmemory_unmap
+
+#### PV domain builder
+
+No stable interfaces, obsolete and can be disabled during QEMU build
+
+ * xc_domain_create
+ * xc_domain_destroy
+ * xc_domain_getinfolist
+ * xc_domain_max_vcpus
+ * xc_domain_setmaxmem (`XEN_DOMCTL_max_mem`)
+ * xc_domain_unpause
+ * xc_evtchn_alloc_unbound
+ * xc_linux_build
+
+## `kexec-tools`
+
+Interface S Underlying Interface S Other users
+---------------------------------- - ------------------------------ - ---------------
+`xc__hypercall_buffer_array_alloc` Should use libxencall
+`xc_hypercall_buffer_array_create` Should use libxencall
+`xc_hypercall_buffer_array_des...` Should use libxencall
+`xc_interface_close`
+`xc_interface_open`
+`xc_get_max_cpus` N `xc_physinfo=SYSCTL_physinfo` N many
+`xc_version` N `XENVER_capabilities` Y libxl,others
+`xc_get_machine_memory_map` N `XENMEM_machine_memory_map` Y libxl
+
+`xc_kexec_exec` N `KEXEC_CMD_kexec` Y None
+`xc_kexec_get_range` N `KEXEC_CMD_kexec_get_range` Y None
+`xc_kexec_load` N `KEXEC_CMD_kexec_load` Y None
+`xc_kexec_unload` N `KEXEC_CMD_kexec_unload` Y None
+
+# Foreign memory mapping functions
+
+The `xc_map_foreign_*` functions were once a bit of a twisty maze, but
+now all are implemented in terms of a single underlying per-OS
+function.
+
+osdep level interfaces:
+
+Interface Linux Minios Netbsd Freebsd Solaris
+-------------------- ----- ------ ------ ------- -------
+`map_foreign_bulk` Y Y compat Y compat
+
+_compat_ means using `osdep_map_foreign_batch` layer
+
+libxenctrl level interfaces:
+
+Interface Underlying interface
+---------------------- ---------------------------
+`xc_map_foreign_pages` `xc_map_foreign_bulk`
+`xc_map_foreign_range` `xc_map_foreign_pages`
+`xc_map_foreign_ranges` `xc_map_foreign_pages`
+`xc_map_foreign_bulk` `osdep: map_foreign_bulk`
+
+`osdep: *` have been collapsed since osdep layer was removed.
+
+`xc_map_foreign_bulk` can be an internal compat helper used by oses
+which have not implemented their own version. It used to use
+`xc_map_foreign_batch` which is now an osdep for those OSes which need
+it.
+
+Interface In-tree users External users
+---------------------- ------------- --------------
+`xc_map_foreign_pages` xenpaging qemu-pv
+`xc_map_foreign_range` xenconsole, kdd, libxl, qemu-dm, qemu-pv
+ mfndump, memshrtool, xenmon,
+ xenstored, xentrace
+`xc_map_foreign_ranges` libxc internal
+`xc_map_foreign_bulk` used recursively qemu-dm
+
+`xc_map_foreign_bulk` should become an underlying stable API.