| Age | Commit message (Collapse) | Author |
|---|
|
This is fairly wide ranging series of patches, fixing multiple compile
errors owing changes in fmtlib 9 and 10.
Care was taken to attempt to not introduce #includes for fmtlib headers
in the public headers of the project, largely through defining a sister
<header file>_fmt.h header that just contains the necessary fmtlib
definitions, along with importing these files into src/... files as
needed. This appears to be the convention (at least, in some parts) of
the upstream codebase.
Briefly outlined, these are issues fixed:
1. Missing fmt::ostream_formatter definitions for types
These used to be implicit, but now fmtlib requires that types
specifically opt into this behavior.
~ src/common/LogEntry.h: clog_type, LogEntry
+ src/include/neorados/RADOS_fmt.hpp: neorados::Object
+ src/include/types_fmt.h: shard_id_t
2. Missing format_to() interface for ceph_le<T>
According to the upstream docs, it should be possible to just use the
explicit cast to the underlying T for formatting, but for whatever
reason, this doesn't work.
Therefore, define a templated format_to() for ceph_le<T> casting to
the underlying T, explicitly.
This is a fmtlib 10 feature -- previously format_to() was reserved for
enums. However, this is the simplest method I can see to avoid needing
to #include fmtlib headers in this header tree.
~ src/include/byteorder.h: ceph_le<T>
3. Make fmt::formatter.format definitions const
Required since fmtlib 8, but seemingly only became an issue in 10.
~ src/osd/osd_types_fmt.h: chunk_info_t, pg_info_t, SnapSet,
ScrubMap::object, ScrubMap
4. Explicitly from snapid_t to uint64_t for fmt::sprintf calls
As the fmt::formatter.parse implementation for snapid_t does not
handle the formatting qualifier used (or any at all).
I think previously this implicitly was casted to uint64_t anyway, so
we're just making the behavior explicit.
~ src/osd/SnapMapper.cc: snapid_t
5. Adding a fmt::formatter implementation for EntityName
Appears to have another casualty in the "no longer try implicit casts"
behavior change of fmtlib 10
~ src/common/LogEntry.h: EntityName
In addition to the above, we defer LogEntry's formatter to the ostream
implementation rather than duplicating the same output, separately in
the fmt::formatter.format definition.
References: https://github.com/fmtlib/fmt/releases/tag/10.0.0
References: https://github.com/fmtlib/fmt/releases/tag/8.0.0
|
|
mostly fixes to line offsets, but there was one real if small change to
a variable init.
|
|
This commit removes the patches to three files from
ceph-17.2.5-fix-iterator-depreciations:
- src/include/xlist.h
- src/include/rados/librados.hpp
- src/crimson/os/seastore/omap_manager/btree/string_kv_node_layout.h
As the upstream has fixed them.
However, the patches to the following have not yet been applied
upstream, and thus remain:
- src/include/rangeset.h
- src/msg/async/dpdk/circular_buffer.h
Upstream-Ref: https://github.com/ceph/ceph/commit/77a3ed56a7b7326814ea5d37214bec1d88ec334e
Upstream-Ref: https://github.com/ceph/ceph/commit/691119c389e119a03aea606b7c4093d46c6db43d
Upstream-Ref: https://github.com/ceph/ceph/commit/905969611d256b6613bd86347bd51ddeb0603b79
Upstream-Ref: https://github.com/ceph/ceph/commit/c662ee698c6366e1b53856eb1a2c903c301a426f
|
|
same patch as before, but context needs updating to apply correctly
|
|
The changes within are now included in the upstream, and are no longer
needed
Upstream-ref: https://github.com/ceph/ceph/pull/50438
|
|
No longer applies to the upstream
|
|
There seems to just be a large amount of effort across the entire
project for this one. Not surprising, given my investigations but it
does make it difficult to provide a singular upstream reference for this
change.
Instead I'll just link directly to all the commits involved since
v17.2.0...
- ceph/ceph@d8d93b0fe3b crimson/osd: correct the fmt::formatter<WatchTimeoutRequest>
- ceph/ceph@71b9344f299 crimson/os: specialize fmt::formatter<> for StagedIterator
- ceph/ceph@32e207da3f3 crimson/os: include fmt/os.h for printing std::error_code
- ceph/ceph@85830863fa4 crimson: specialize fmt::formatter<> for formatting using {fmt}
- ceph/ceph@76e4c5d83c2 test/librados/aio_cxx: use fmt formatting
- ceph/ceph@2e7aea7b0fd crimson/os: print pointer using fmt::ptr()
- ceph/ceph@c9569f8eb02 crimson: specialize fmt::formatter<>
- ceph/ceph@01f6bef6e52 *: specialize fmt::formatter<> for formatting using {fmt}
- ceph/ceph@3f67d67ede1 crimson: specialize fmt::formatter<> to print using {fmt}
- ceph/ceph@42dc970392e crimson/net: print shared_ptr using fmt::ptr()
- ceph/ceph@fe1e6b6c4eb neorados: compile with fmt v9
- ceph/ceph@b2c61113ffb crimson/net: define fmt::formatter for write_state_t
- ceph/ceph@1f1398bc515 mon: Fix snprintf truncation warning by switching to fmtlib
- ceph/ceph@93dfb7e88b9 common: improving fmtlib handling of ceph::utime_t
- ceph/ceph@3d7a895d197 include/object_fmt: mark fmt::formatter<>::format() const
- ceph/ceph@2901943618f common/LogEntry: specialize fmt::formatter<LogEntry>
- ceph/ceph@6e223a68021 common/Journald: include msg/msg_fmt.h
- ceph/ceph@c0a3d460297 mon: use fmt::format for stderr cluster logging
- ceph/ceph@cef04bc4ed0 common: use fmt::format for stderr logging
- ceph/ceph@31aa539f266 osd: use fmt/ranges.h for printing ranges
- ceph/ceph@b7b7ff4cce3 include/types_fmt: use fmt::join() to format containers
- ceph/ceph@b287fadde3e fmt: pickup v8.1.1
- ceph/ceph@3c46dd64bbc cmake: bump up required fmt version
- ceph/ceph@01f9dc84826 msg/msg_type: use fmt::format to print entity_addr
- ceph/ceph@ac114ad172b crimson/os: use fmt::format() instead of sstream
- ceph/ceph@bb9d099ede8 crimson: specialize fmt::formatter<>() for crimson types
- ceph/ceph@3fd91d7e6d9 crimson/os/seastore: drop compile-time fmt validation
|
|
Underlying issue found in f5af709 has been fixed upstream, re-enabling
test.
Upstream-Ref: https://github.com/ceph/ceph/commit/b5218528d63ae87ca77229bce1635c62411959e8
|
|
This was a temporary fix for a v17 rebuild. The upstream has fixed
many, many fmt issues in the new release.
Reverts: 7dfdbe1
Reverts: 34d11c3
|
|
As the check_generated.sh test now depends on xmlstarlet, and
`hostname` is used a bunch.
|
|
required by the upstream now
|
|
Mostly updating the pkgname array to reflect the refactoring done on
packaging, and a slight change in field ordering to move pkgdesc higher
in the file
|
|
Improve formatting, and breakup into groups (system and python,
currently), so as to escape the giant word soup that currently exists.
|
|
These are virtual (dependency only) packages that serve a couple
purposes:
1. Backwards compatibility with the existing Archlinux ceph packages.
That is:
- ceph-libs: all non-binary packages
- ceph-mgr: (same as before / not virtual)
- ceph: all binary packages
2. Stable targets for certain functionality, namely cluster components
and the `ceph` CLI utility which is used for most interactions with
a cluster.
- ceph-libs
- ceph-cluster
- ceph-cli
- ceph
|
|
These are mostly used by the upstream development team for interacting
with various internals of the cluster and live tests.
However, the impetus for this entire patchset also lurks: java-cephfs.
A package that requires we pull in the entire java runtime, likely for a
package absolutely no one uses. Good riddance.
- ceph-tools
- ceph-test
- java-cephfs
|
|
These are a couple useful utilities for interfacing with ceph, of which
ceph-volume is likely the most important, as is it responsible for
provisioning devices for use as OSDs.
cephadm is an honorable mention, being a 9k line python file used in
managing clusters... if you like the idea of having 9k lines of python
sitting between you and your entire storage backplane.
(I do not)
- ceph-volume
- cephfs-shell
- cephfs-top
- ceph-cephadm
|
|
These comprise the daemons and associated tooling that form the cluster
or "backend" of a ceph deployment.
Notably, they have a dependencies on both the core libs, python libs
_and_ librgw (a client), as despite being a backend or server component;
ceph-rgw is _actually_ a client of ceph (and librgw specifically), that
just so happens to run as a daemon -- providing object storage.
- ceph-base
- ceph-mon
- ceph-mgr
- ceph-osd
- ceph-mds
- ceph-rgw
|
|
These are python bindings to the underlying C/C++ libs, and are used for
most of the ancillary CLI tooling ceph provides, including as support
tooling for cluster components.
- python-ceph-common
- python-rados
- python-rbd
- python-cephfs
- python-rgw
|
|
These form the core clients for ceph; rbd (block), cephfs
(filesystem), rgw (object).
- librbd
- ceph-rbd
- libcephfs
- ceph-cephfs
- librgw
|
|
These are the core / common libraries that almost everything else will
depend on at some level.
libcephsqlite is an outlier, but is also depended on by other ceph
components despite _technically_ being a RADOS client, hence its
inclusion.
- ceph-compressor
- ceph-crypto
- ceph-erasure
- ceph-common
- librados
- ceph-rados
- libcephsqlite
|
|
The true workhorse for packaging ceph, this function is responsible for
calling make install, and _ensuring all files accounted for_ -- moved or
deleted -- before yielding.
It will contain a series of `_package` invocations, one per package,
with each actual package function being a thin wrapper around:
'mv __pkg__/$pkgname/* "$pkgdir"'
|
|
These are the beginning of a refactoring of how we handle the package_*
functions.
Currently, we run `make install` once per packaging function and attempt
to delete the bits that aren't related to the package currently running.
This has several downsides, including conflicting file owners, missed
addition(al file)s in later versions and speed.
These new functions will serve as the engine for enumerating the precise
file dependencies for each package, and provide a pretty output during
the packaging process of this list.
|
|
|
|
* HEAD
| \
| * 2821569 <github@luxolus.com> pkgbuild: pkgrel=4
| /
| master
Signed-off-by: Bazaah <github@luxolus.com>
|
|
Rebuild for boost, thrift and some python stuff
|
|
* HEAD
| \
| * 7692995 <github@luxolus.com> pkgbuild: pkgrel=3
| * 5c53c4e <github@luxolus.com> pkgbuild: use ceph-17.2.6-cython-fixes.patch
| * 34d11c3 <github@luxolus.com> pkgbuild: use backport-with-fmt-version.patch
| * 22f7a39 <github@luxolus.com> repo: add ceph-17.2.6-cython-fixes.patch
| * 7dfdbe1 <github@luxolus.com> repo: add backport-with-fmt-version.patch
| /
| master
Signed-off-by: Bazaah <github@luxolus.com>
|
|
|
|
|
|
Issue: https://github.com/bazaah/aur-ceph/issues/17
Reported-by: https://aur.archlinux.org/account/UweSauter
Reported-in: https://aur.archlinux.org/pkgbase/ceph#comment-926651
|
|
cython v3.0.0 introduced a breaking change to exception propagation
through cdef (and ctypedef) functions, creating a type error when trying
to assign &no_op_progress_callback = librbd_progress_fn_t, and similar
for __aio_complete_cb = rbd_callback_t
This fixes those. Surprisingly these are the only compile errors I found
for this pattern, because there are many more ctypedef'd fptrs in the
pybind stuff.
References: https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html#exception-values-and-noexcept
|
|
This is a workaround for the fmtlib v10 removal of
FMT_DEPRECATED_OSTREAM. v18 has (many, many) fixes for this, but it
simply not feasible to backport all of them, particularly because v18 is
releasing shortly.
So instead, we use the bundled fmtlib rather than the system one, for
now.
Of special note, we add the EXACT find_package attr, so that cmake is
forced to conclude that there isn't a compatible system version. This is
slightly uglier than messing with cmake version range specs, but it
requires much less thought for the same outcome, and is only a temp fix.
Upstream-ref: https://github.com/ceph/ceph/commit/051ec194df16927187cbe4b26dacd06c31fa186a
Upstream-ref: https://github.com/ceph/ceph/commit/0d41777cea1720cdf3fc54e182325b4440b08d40
|
|
* HEAD
| \
| * 64699fc <github@luxolus.com> pkgbuild: pkgrel=2
| * 800578f <github@luxolus.com> pkgbuild: use cherrypy, pylint patches
| * 6741cd4 <github@luxolus.com> repo: mgr/dashboard deps pylint->2.17, cherrypy->18
| * 887c85e <github@luxolus.com> pkgbuild: use mgr-dashboard-node-version.patch
| * d931804 <github@luxolus.com> repo: add mgr-dashboard-node-version.patch
| * abe5a35 <github@luxolus.com> pkgbuild: use gcc13 patches
| * aab480f <github@luxolus.com> repo: patches for gcc13 includes in rocksdb, ceph
| /
| master
Fixes: #12
Fixes: #14
Signed-off-by: Bazaah <github@luxolus.com>
|
|
|
|
|
|
cherrypy is a runtime dependency, while pylint is a build/lint dep, both
of them encountered import errors due to API removals in py3.11, see the
links for more.
pylint's is indirect, a dependency of a dependency:
pylint->astroid->wrapt
but fixed by moving to a more recent version
- ceph-17.2.6-mgr-dashboard-cherrypy-18.patch
- ceph-17.2.6-mgr-dashboard-pylint-217.patch
Issue: https://github.com/bazaah/aur-ceph/issues/12
References: https://github.com/cherrypy/cherrypy/commit/8245a74aa4e090c40445535a9ce3997ed9904798
References: https://github.com/GrahamDumpleton/wrapt/issues/196
|
|
|
|
up the bundled node version to 16, so as to fix node-gyp errors
originating from a too-old npm version in 14.
Basically, python 3.11 removed a deprecated file open option from
python, which consequently blows up `npm ci`, preventing the build.
Issue: https://github.com/bazaah/aur-ceph/issues/12
References: https://github.com/npm/cli/issues/5113
References: https://github.com/npm/cli/issues/4664
References: https://github.com/npm/cli/issues/4660
|
|
|
|
These are caused by the changes in gcc13, that move previously
included-by-default imports like uint64_t and system_error into separate
headers.
Parts of the rocksdb patch where taken from the upstream --
facebook/rocksdb@88edfbf, but more were needed for however ceph
configures the build.
We also add a couple that were missing in ceph itself.
- ceph-17.2.6-ceph-gcc13-includes.patch
- ceph-17.2.6-rocksdb-gcc13-includes.patch
References: https://gcc.gnu.org/gcc-13/porting_to.html#header-dep-changes
References: https://github.com/facebook/rocksdb/commit/88edfbfb5e1cac228f7cc31fbec24bb637fe54b1
|
|
* HEAD
| \
| * 75c66e6 <github@luxolus.com> pkgbuild: pkgver=17.2.6, pkgrel=1
| * 55f267d <github@luxolus.com> pkgbuild: rm disable-test-cls-fifo.patch
| * 3a4ece4 <github@luxolus.com> pkgbuild: revert ceph-18.0.0-mgr-sqlite-pragmas.patch
| * 1bd9308 <github@luxolus.com> pkgbuild: revert rdkafka.mpatch, use therein
| * 7f61f1b <github@luxolus.com> pkgbuild: rere tox-mypy-false-postive.patch
| /
| master
Signed-off-by: Bazaah <github@luxolus.com>
|
|
|
|
Upstream appears to have removed the test
Upstream-Ref: https://github.com/ceph/ceph/commit/62121704f019f225b1df4b0c63ba2d89fe9be9ca
|
|
As this is now backported in the upstream
Upstream-Ref: https://github.com/ceph/ceph/commit/b49753e374e0ba30e61eab0b7d88abb5104e5975
Reverts: 82a7efacca33c5640103622e251755a5523e78cd
|
|
No longer needed, as the Archlinux package has fixed this issue by
reverting to an older (?) method for building rdkafka, which doesn't
contain the bug this patch worked around.
Reverts: dd2565ded777ce438f5063d6e9cc8b2fe22244e7
Reverts: d0451743748d9c381a598fb45b658c3669ee357f
References: https://github.com/archlinux/svntogit-community/commit/924fddef74a86f020ec6813291979946f6da55da
|
|
Same patch, just updated to follow upstream changes to
src/pybind/mgr/tox.ini
|
|
* HEAD
| \
| * 948bd85 <github@luxolus.com> pkgbuild: pkgrel=6
| * d470b03 <github@luxolus.com> pkgbuild: use ceph-17.2.5-rgw-client-boost-string-view.patch
| * 5bc29e5 <github@luxolus.com> pkgbuild: add ceph-17.2.5-rgw-client-boost-string-view.patch
| * dd2565d <github@luxolus.com> pkgbuild: use rdkafka.mpatch + set env
| * d045174 <github@luxolus.com> pkgbuild: add rdkafka.mpatch
| /
| master
Fixes: #11
Signed-off-by: Bazaah <github@luxolus.com>
|
|
|
|
|
|
So long story short, boost 1.81 has fairly large breaking API changes,
as they migrated to use an internal string_view shim rather than
std::string_view, across the entire beast codebase. Therefore, we
remove any attempts to convert to std::string_view from
boost::core::string_view, as it implicitly handles conversions to
std::string where needed (like std::string_view).
Long story. boostorg/beast#2451 introduced a change in their string_view
type defs for beast, which ultimately removed the to_string() method
inherited from boost::string_view (which is different to boost::core),
effectively changing the type of most of the returned values in
rgw_asio_client.cc's header manipulation logic.
However, this new shim string_view implicitly converts to std::string,
which is needed by RGWEnv::set, hence our removal of the various
to_string()s scattered across the file.
Notably, RGWEnv::set is remarkably strict in its accepted values, so I
trust that if this compiles we have introduced a subtle use after free
vis-a-vis all these string_views flying around.
References: https://github.com/boostorg/beast/pull/2451
References: https://github.com/ceph/ceph/blob/v17.2.5/src/rgw/rgw_asio_client.cc
References: https://github.com/boostorg/beast/issues/2594#issuecomment-1360956011
References: https://github.com/ceph/ceph/blob/v17.2.5/src/rgw/rgw_env.cc#L22
|
|
|