libpathrs v0.2.1 -- "やられたらやり返す。倍返しだ！"

This is a minor release of libpathrs which includes some fairly critical
bugfixes, as well as some Rust-specific packaging fixes that should make
it easier to use libpathrs in Rust projects.

Security

• When using ProcfsHandle::open_follow on non-magic-link symlinks,
  libpathrs could fall victim to an overmount attack because we had incorrectly
  assumed that opening a symlink as a final component would be "atomic" (this
  is only true for magic-links, which was the primary usecase we had in mind
  for this API).

  We now try to use the safe procfs resolver even on symlinks to handle the
  "regular symlink case". Note that (due to a separate bug in
  ProcfsHandle::open_follow that has also been fixed), privileged users would
  likely still get an error in this case.

Added

• Error and ErrorKind now have a can_retry helper that can be used to
  make retry loops easier for callers.
• libpathrs now has fairly comprehensive end-to-end tests for all our bindings
  (written in a language-agnostic way), to ensure correctness and uniformty
  when you use libpathrs, no matter which language you use.

Fixed

• python bindings: fix pathrs.procfs examples in README.
• go bindings: fix the internal os.FileMode to S_IF* conversion to not
  auto-include S_IFREG for non-Mknod operations (previously this would
  cause MkdirAll to error out).
• Root::create_file now supports O_TMPFILE.
• Previously, trying to use ProcfsHandle::open_follow with a path whose
  penultimate component was a symlink (i.e.,
  ProcfsHandle::open_follow(ProcfsBase::ProcRoot, "self/status")) would
  result in an error due to a mistake in how we handled looking up parent
  directories. This has been fixed, and this will now work the way you expect
  (though you should still use ProcfsBase::ProcSelf instead in the above
  example).
• ProcfsHandle::open_follow was missing the logic to temporarily allocate a
  non-subset=pid if the target does not exit. This ended up accidentally
  mitigating the ProcfsHandle::open_follow security issue mentioned above
  (for fsopen(2) users trying to open symlinks in ProcfsBase::ProcRoot --
  note that only ProcfsBase::ProcRoot contains such symlinks in the first
  place).
• Quite a few Root operations that required resolving the parent directory of
  the user-provided path could crash if passed / or return an unhelpful error
  when passed .. We now return a proper error in these cases.

Changed

• The openat2 resolver will now return -EAGAIN if the number of openat2
  retries is exceeded -- this allows higher-level users easily detect if an
  error is an indication they should retry (based on their own retry policy).

  In addition, the number of retries done has been bumped from 32 to 128
  based on some benchmarking which showed that 32 could fail up to 3% of the
  time but 128 would only fail ~0.1% of the time in the worst case scenario
  of an attacker that can saturate all cores with rename(2) operations.

  Users that need stronger resiliency guarantees can do their own additional
  retry loop on top of libpathrs by checking the return value for EAGAIN.
  Please note that we would strongly recommend having some restriction to avoid
  denial-of-service attacks (such as a deadline -- for reference, our testing
  showed that even with >50k trials containing >200k operations a deadline of
  1ms was never exceeded even in the most pessimistic attack scenario).

• The O_PATH resolver for ProcfsHandle will now return ELOOP for
  magic-links that look like foo:[bar] in order to better match openat2(2)
  (examples include anon_inode, nsfs, pipe, and other such special
  inodes). Previously we would just return ENOENT.

Signed-off-by: Aleksa Sarai [email protected]

libpathrs v0.2.0 -- "You're gonna need a bigger boat."

This is the first release of libpathrs in the past year and contains a
very large number of improvements and changes since 0.1.3. While
libpathrs 0.1.3 was functional, this update makes it far more usable for
most users.

At this point we do not expect the Rust API to change significantly
(though we do expect some more breaking changes before 1.0). Users of
libpathrs.so should also be aware that this release adds symbol
versioning, which should allow for more fearless upgrading in the future
-- changes to the C API will now include backward-compatible functions
for programs compiled with old symbol versions.

Note

As of this release, the libpathrs repository has been moved to
https://github.com/cyphar/libpathrs. Please update any references you have
(though GitHub will redirect the old repository name to the new one).

In addition, the go-pathrs package has been moved to a vanity URL of
cyphar.com/go-pathrs. Please update your Go import paths accordingly.

Important

The license of this project has changed. Now, the entire project (including
all language bindings and examples) is licensed under the terms of the
Mozilla Public License version 2.0. Additionally, the Rust crate (and
cdylib) may also be used (at your option) under the terms of the GNU Lesser
General Public License version 3 (or later).

For more information, see COPYING.md and the "License"
section of the README.

The long-term plan is to restructure the project so that src/capi is a
separate crate that is only licensed as GNU LGPLv3+ and the Rust crate is
only licensed under as MPLv2, but for technical reasons this is difficult to
achieve at the moment. The primary purpose for dual-licensing is to try to
assuage possible concerns around the GNU LGPLv3 requirements to be able to
"recombine or relink the Application with a modified version of the Linked
Version to produce a modified Combined Work" in the context of the Rust build
system, while also allowing us to license the cdylib portion under the GNU
LGPLv3+.

Breaking

• python bindings: Root.creat has had its filemode and flags arguments
  swapped to match the argument order of openat2 (and Root.creat_raw). This
  also now makes filemode have a default value of 0o644 if unspecified.
• Most of the C FFI functions have been renamed:
  • Operations on a Root have been renamed to have a pathrs_inroot_ prefix.
  • pathrs_root_open has been renamed to pathrs_open_root, to avoid
    confusion with pathrs_inroot_* functions and clarify what it is opening.
  • However, libpathrs.so now uses symbol versioning and so (mostly as a
    proof-of-concept) programs compiled against libpathrs 0.1 will continue to
    function with libpathrs 0.2.
• python bindings: Root.open has been changed to be a wrapper of
  pathrs_inroot_open instead of being a wrapper around the Root
  constructor.
• All C FFI functions that return a file descriptor now set O_CLOEXEC by
  default. Previously some functions that took O_* flags would only set
  O_CLOEXEC if the user explicitly requested it, but O_CLOEXEC is easy to
  unset on file descriptors and having it enabled is a more sane default.
• The C API values of the pathrs_proc_base_t enum (PATHRS_PROC_BASE_*) have
  different values, in order to support ProcfsBase::ProcPid passing from C
  callers. Any binaries compiled with the old headers will need to be
  recompiled to avoid spurious behaviour.
  • This required a breaking change in the Go bindings for libpathrs.
    ProcfsBase is now an opaque struct type rather than a simple int
    wrapper -- this was necessary in order to add support for
    ProcfsBase::ProcPid in the form of the ProcBasePid helper function.
• go bindings: the Go module name for the libpathrs Go bindings has been
  renamed to cyphar.com/go-pathrs. This will cause build errors for existing
  users which used the old repository path, but can easily be fixed by updating
  go.mod and go.sum to use the new name.
• go bindings: the procfs APIs have been moved to a procfs subpackage, and
  several of the exported types and functions have changed names. We have not
  provided any compatibility aliases.
• python bindings: the procfs APIs have been moved to a procfs submodule. We
  have not provided any compatibility aliases.

Added

• python bindings: add Root.creat_raw to create a new file and wrap it in a
  raw WrappedFd (os opposed to Root.creat which returns an os.fdopen).

• Root: it is now possible to open a file in one shot without having to do an
  intermediate resolve step with Root::open_subpath. This can be more
  efficient in some scenarios (especially with the openat2-based resolver or
  for C FFI users where function calls are expensive) as it saves one file
  descriptor allocation and extra function calls.

• Error: ErrorKind is now exported, allowing you to programmatically handle
  errors returned by libpathrs. This interface may change in the future (in
  particular, ErrorKind::OsError might change its representation of errno
  values).

• capi: errors that are returned by libpathrs itself (such as invalid arguments
  being passed by users) will now contain a saved_errno value that makes
  sense for the error type (so ErrorKind::InvalidArgument will result in an
  EINVAL value for saved_errno). This will allow C users to have a nicer
  time handling errors programmatically.

• tests: we now have a large array of tests for verifying that the core lookup
  logic of libpathrs is race-safe against various attacks. This is no big
  surprise, given libpathrs's design, but we now have more extensive tests than
  github.com/cyphar/filepath-securejoin.

• procfs: added ProcfsBase::ProcPid(n) which is just shorthand when operating
  on a operating on a different process. This is also now supported by the C
  API (by just passing the pid_t instead of a special pathrs_proc_base_t
  value).

• procfs: we now make use of /proc/thread-self/fdinfo's mnt_id field to try
  to thwart bind-mount attacks on systems without STATX_MNT_ID support.

  On systems with openat2(2), this protection is effectively just as safe as
  STATX_MNT_ID (which lets us lower the minimum recommended kernel version
  from Linux 5.8 to Linux 5.6). For older systems, this protection is not
  perfect, but is designed to be difficult for an attacker to bypass as
  consistently and easily as it would be without these protections.

  Note that it is still the case that post-6.8 kernels (STATX_MNT_ID_UNIQUE)
  are still the most strongly recommended kernels to use.

• procfs: ProcfsHandle is now ProcfsHandleRef<'static>, and it is now
  possible to construct borrowed versions of ProcfsHandleRef<'fd> and still
  use them. This is primarily intended for our C API, but Rust users can make
  use of it if you wish. It is possible we will move away from a type alias for
  ProcfsHandle in the future.

• capi: All of that pathrs_proc_* methods now have a pathrs_proc_*at
  variant which allows users to pass a file descriptor to use as the /proc
  handle (effectively acting as a C version of ProcfsHandleRef<'fd>). Only
  users that operate heavily on global procfs files are expected to make use of
  this API -- the regular API still lets you operate on global procfs files.
  Users can pass PATHRS_PROC_DEFAULT_ROOTFD (-EBADF) as a file descriptor
  to use the cached API (the old API methods just do this internally).

• procfs: a new ProcfsHandleBuilder builder has been added to the API, which
  allows users to construct an unmasked (i.e., no-subset=pid) ProcfsHandle.

  This should only be used sparingly and with great care to avoid leaks, but it
  allows some programs to amortise the cost of constructing a procfs handle
  when doing a series of operations on global procfs files (such as configuring
  a large number of sysctls).

  We plan to add a few more configuration options to ProcfsHandleBuilder in
  the future, but ProcfsHandleBuilder::unmasked will always give you an
  unmasked version of /proc regardless of any new features.

• procfs: ProcfsHandleRef can now be converted to OwnedFd with
  .into_owned_fd() (if it is internally an OwnedFd) and borrowed as
  BorrowedFd with AsFd::as_fd. Users should take great care when using the
  underlying file descriptor directly, as using it opens you up to all of the
  attacks that libpathrs protects you against.

• capi: add pathrs_procfs_open method to create a new ProcfsHandle with a
  custom configuration (a-la ProcfsHandleBuilder). As with
  ProcfsHandleBuilder, most users do not need to use this.

  • python bindings: ProcfsHandle wraps this new API, and you can construct
    custom ProcfsHandles with ProcfsHandle.new(...).
    ProcfsHandle.cached() returns the cached global ProcfsHandle. The
    top-level proc_* functions (which may be removed in future versions) are
    now just bound methods of ProcfsHandle.cached() and have been renamed to
    remove the proc_ prefix (now that the procfs API lives in a separate
    pathrs.procfs module).
  • go bindings: ProcfsHandle wraps this new API, and you can construct a
    custom ProcfsHandles with OpenProcRoot (calling this with no arguments
    will produce the global cached handle if the handle is being cached). The
    old Proc* functions have been removed entirely.

• capi: We now use symbol versioning for libpathrs.so, which should avoid
  concerns about future API breakages. I have tested all of the key aspects of
  this new symbol versioning setup and it seems Rust provides everything
  necessary (when testing this last year, I was unable to get
  back...

libpathrs v0.1.3 -- "自動化って物は試しとすればいい物だ"

This is the third patch release of the 0.1.z branch of libpathrs. This
only includes a few small fixes, but is mainly intended to just test
that our automated release GitHub Actions correctly work on a real
release in the main repo.

• gha: our Rust crate and Python bindings are now uploaded automatically from a
  GitHub action when a tag is pushed.

• syscalls: the pretty-printing of openat2 errors now gives a string
  description of the flags passed rather that just a hex value (to match other
  syscalls).

• python bindings: restrict how our methods and functions can be called using
  / and * to reduce the possibility of future breakages if we rename or
  re-order some of our arguments.

Thanks to the following people who made this release possible:

• Aleksa Sarai [email protected]

Signed-off-by: Aleksa Sarai [email protected]

libpathrs v0.1.2 -- "蛇のように賢く、鳩のように素直でありなさい"

This is the second patch release of the 0.1.z branch of libpathrs. The
primary fixes are related to the Python libpathrs bindings.

• python bindings: add a minimal README for PyPI.
• python bindings: actually export PROC_ROOT.
• python bindings: add type annotations and py.typed to allow for downstream
  users to get proper type annotations for the API.

Thanks to the following people who made this release possible:

• Aleksa Sarai [email protected]

Signed-off-by: Aleksa Sarai [email protected]

libpathrs v0.1.1 -- "頒布と聞いたら蛇に睨まれた蛙になるよ"

This is the first patch release of the 0.1.z branch of libpathrs. The
primary fixes are related to packaging the Python libpathrs bindings for
distributions.

• procfs: add support for operating on files in the /proc root (or other
  processes) with ProcfsBase::ProcRoot.

  While the cached file descriptor shouldn't leak into containers (container
  runtimes know to set PR_SET_DUMPABLE, and our cached file descriptor is
  O_CLOEXEC), I felt a little uncomfortable about having a global unmasked
  procfs handle sitting around in libpathrs. So, in order to avoid making a
  file descriptor leak by a libpathrs user catastrophic, libpathrs will
  always try to use a "limited" procfs handle as the global cached handle
  (which is much safer to leak into a container) and for operations on
  ProcfsBase::ProcRoot, a temporary new "unrestricted" procfs handle is
  created just for that operartion. This is more expensive, but it avoids a
  potential leak turning into a breakout or other nightmare scenario.

• python bindings: The cffi build script is now a little easier to use for
  distributions that want to build the python bindings at the same time as the
  main library. After compiling the library, set the PATHRS_SRC_ROOT
  environment variable to the root of the libpathrs source directory. This
  will instruct the cffi build script (when called from setup.py or
  python3 -m build) to link against the library built in the source directory
  rather than using system libraries. As long as you install the same library
  later, this should cause no issues.

  Standard wheel builds still work the same way, so users that want to link
  against the system libraries don't need to make any changes.

• Root::mkdir_all no longer does strict verification that directories craeted
  by mkdir_all "look right" after opening each component. These checks didn't
  protect against any practical attack (since an attacker could just get us to
  use a directory by creating it before Root::mkdir_all and we would happily
  use it) and just resulted in spurious errors when dealing with complicated
  filesystem configurations (POSIX ACLs, weird filesystem-specific mount
  options). (#71)

• capi: Passing invalid pathrs_proc_base_t values to pathrs_proc_* will now
  return an error rather than resulting in Undefined Behaviour™.

Thanks to the following people who made this release possible:

• Aleksa Sarai [email protected]
• Ayose [email protected]

Signed-off-by: Aleksa Sarai [email protected]

libpathrs v0.1.0 -- "負けたくないことに理由って要る?"

This is the first fully-fledged release of libpathrs. All of the obvious
features have been implemented, and both the C and Rust APIs have been
redesigned a few times so I am fairly sure this design is pretty solid
now. While there was a fairly long hiatus in development, we're back on
track.

Many things have changed since the last release, here are the highlights:

• cffi: Redesign the entire API to be file descriptor based, removing the need
  for complicated freeing logic and matching what most kernel APIs actually
  look like. While there is a risk that users would operate on file descriptors
  themselves, the benefits of a pure-fd-based API outweigh those issues (and
  languages with bindings like Python and Go can easily wrap the file
  descriptor to provide helper methods and avoid this mistake by users).

  Aside from making C users happier, this makes writing bindings much simpler
  because every language has native support for handling the freeing of file
  objects (Go in particular has *os.File which would be too difficult to
  emulate outside of the stdlib because of it's unique Close handling).

  • Unfortunately, this API change also removed some information from the C API
    because it was too difficult to deal with:
    • Backtraces are no longer provided to the C API. There is no plan to
      re-add them because they complicate the C API a fair bit and it turns out
      that it's basically impossible to graft backtraces to languages that have
      native backtrace support (Go and Python) so providing this information
      has no real benefit to anyone.
    • The configuration API has been removed for now. In the future we will
      probably want to re-add it, but figuring out a nice API for this is left
      for a future (pre-1.0) release. In practice, the default settings are the
      best settings to use for most people anyway.

• libpathrs now has a "safe procfs resolver" implementation that
  verifies all of our operations on /proc are done safely (including
  using fsopen(2) or open_tree(2) to create a private /proc to protect
  against race attacks).

  This is mainly motivated by issues like CVE-2019-16884 and
  CVE-2019-19921, where an attacker could configure a malicious mount
  table such that naively doing /proc operations could result in
  security issues. While there are limited things you can do in such a
  scenario, it is far more preferable to be able to detect these kinds
  of attacks and at least error out if there is a malicious /proc.

  This is based on similar work I did in filepath-securejoin.

  • This API is also exposed to users through the Rust and C FFI because
    this is something a fair number of system tools (such as container
    runtimes) need.

• libpathrs now has an official MSRV of 1.63, which is verified by our
  CI. The MSRV was chosen because it's the Rust version in Debian stable
  and it has io_safety which is one of the last bits we absolutely need.

• root: new Root methods:

  • readlink and resolve_nofollow to allow users to operate on symlinks
    directly (though it is still unsafe to use the returned path for
    lookups!).
  • remove_all so that Go users can switch from os.RemoveAll (though
    Go's os.RemoveAll is safe against races since Go 1.21.11 and Go
    1.22.4).
  • mkdir_all so that Go users can switch from os.MkdirAll. This is
    based on similar work done in filepath-securejoin.

• root: The method for configuring the resolver has changed to be more
  akin to a getter-setter style. This allows for more ergonomic usage
  (see the RootRef::with_resolver_flags examples) and also lets us avoid
  exposing internal types needlessly.

  As part of this change, the ability to choose the resolver backend was
  removed (because the C API also no longer supports it). This will
  probably be re-added in the future, but for now it seems best to not
  add extra APIs that aren't necessary until someone asks.

• opath resolver: We now emulate fs.protected_symlinks when resolving
  symlinks using the emulated opath resolver. This is only done if
  fs.protected_symlinks is enabled on the system (to mirror the
  behaviour of openat2).

• packaging: Add an autoconf-like install.sh script that generates a
  pkg-config specification for libpathrs. This should help distributions
  package libpathrs.

• Handling of // and trailing slashes has been fixed to better match what
  users expect and what the kernel does.

• opath resolver: Use reference counting to avoid needlessly cloning files
  internally when doing lookups.

• cffi: Building the C API is now optional, so Rust crates won't contain any of
  the C FFI code and we only build the C FFI crate types manually in the
  makefile. This also lets us remove some dependencies and other annoying
  things in the Rust crate (since those things are only needed for the C API).

• python bindings: Switch to setuptools to allow for a proper Python package
  install. This also includes some reworking of the layout to avoid leaking
  stuff to users that just do import pathrs.

• bindings: All of the bindings were rewritten to use the new API.

• rust: Rework libpathrs to use the (stabilised in Rust 1.63) io_safety
  features. This lets us avoid possible "use after free" issues with file
  descriptors that were closed by accident.

  This required the addition of HandleRef and RootRef to wrap BorrowedFd
  (this is needed for the C API, but is almost certainly useful to other
  folks). Unfortunately we can't implement Deref so all of the methods need
  to be duplicated for the new types.

• Split Root::remove into Root::remove_file (unlink) and
  Root::remove_dir (rmdir) so we don't need to do the retry loop anymore.
  Some users care about what kind of inode they're removing, and if a user
  really wants to nuke a path they would want to use Root::remove_all anyway
  because the old Root::remove would not remove non-empty directories.

Thanks to the following contributors who made this release possible:

• Aleksa Sarai [email protected]
• Jakub Wilk [email protected]
• Toru Komatsu [email protected]

Signed-off-by: Aleksa Sarai [email protected]

libpathrs 0.0.2

The main improvements in this release are:

• Go bindings (thanks to Maxim).
• Massively reworked threading model for C FFI -- now all errors are
  thread-local but libpathrs objects can be used from multiple threads
  without races.
• pathrs_from_fd() now makes a copy of the input file to avoid issues
  with languages that might not be able to smoothly handle the concept
  of "forget this file you used to manage".
• Update to the latest (now-merged) openat2 syscall ABI.

Thanks to all the contributors who made this release possible:

• Aleksa Sarai [email protected]
• Maxim Zhiburt [email protected]

Signed-off-by: Aleksa Sarai [email protected]

libpathrs 0.0.1

First PoC release, but still lots of work to be done.

Signed-off-by: Aleksa Sarai [email protected]

libpathrs 0.0.0 [yanked]

Historic release which had broken docs and was yanked.

Signed-off-by: Aleksa Sarai [email protected]