GH-51007: [C++] Make uriparser an external dependency - #51244
Conversation
| packages+=("${MINGW_PACKAGE_PREFIX}-gflags") | ||
| packages+=("${MINGW_PACKAGE_PREFIX}-grpc") | ||
| packages+=("${MINGW_PACKAGE_PREFIX}-gtest") | ||
| packages+=("${MINGW_PACKAGE_PREFIX}-uriparser") |
There was a problem hiding this comment.
Can we revert this because we already have it in this list?
| if(URIPARSER_ROOT) | ||
| find_library(URIPARSER_LIB | ||
| NAMES uriparser | ||
| PATHS ${URIPARSER_ROOT} | ||
| PATH_SUFFIXES ${ARROW_LIBRARY_PATH_SUFFIXES} | ||
| NO_DEFAULT_PATH) | ||
| find_path(URIPARSER_INCLUDE_DIR | ||
| NAMES uriparser/Uri.h | ||
| PATHS ${URIPARSER_ROOT} | ||
| NO_DEFAULT_PATH | ||
| PATH_SUFFIXES ${ARROW_INCLUDE_PATH_SUFFIXES}) | ||
| else() |
There was a problem hiding this comment.
We don't need this check. We need to check only CMake package and pkg-config package.
| NO_DEFAULT_PATH | ||
| PATH_SUFFIXES ${ARROW_INCLUDE_PATH_SUFFIXES}) | ||
| else() | ||
| find_package(PkgConfig QUIET) |
There was a problem hiding this comment.
Could you add QUIET only when uriparserAlt_FIND_QUIETLY?
| PATH_SUFFIXES ${ARROW_INCLUDE_PATH_SUFFIXES}) | ||
| else() | ||
| find_package(PkgConfig QUIET) | ||
| pkg_check_modules(URIPARSER_PC liburiparser) |
There was a problem hiding this comment.
Could you add QUIET when uriparserAlt_FIND_QUIETLY?
Could you add IMPORTED_TARGET?
| NO_DEFAULT_PATH | ||
| PATH_SUFFIXES ${ARROW_LIBRARY_PATH_SUFFIXES}) | ||
| set(URIPARSER_VERSION "${URIPARSER_PC_VERSION}") | ||
| else() |
There was a problem hiding this comment.
We don't need this because we want to use CMake target instead of variables.
There was a problem hiding this comment.
We don't need this because we want to use CMake target instead of variables.
Didnt really get this. I have made some changes , please see if they work.
| find_library(URIPARSER_LIB | ||
| NAMES uriparser | ||
| PATH_SUFFIXES ${ARROW_LIBRARY_PATH_SUFFIXES}) | ||
| find_path(URIPARSER_INCLUDE_DIR | ||
| NAMES uriparser/Uri.h | ||
| PATH_SUFFIXES ${ARROW_INCLUDE_PATH_SUFFIXES}) |
| externalproject_add(uriparser_ep | ||
| ${EP_COMMON_OPTIONS} | ||
| CMAKE_ARGS ${URIPARSER_CMAKE_ARGS} | ||
| INSTALL_DIR ${URIPARSER_PREFIX} | ||
| URL ${ARROW_URIPARSER_SOURCE_URL} | ||
| URL_HASH "SHA256=${ARROW_URIPARSER_BUILD_SHA256_CHECKSUM}" | ||
| BUILD_BYPRODUCTS "${URIPARSER_STATIC_LIB}") |
There was a problem hiding this comment.
Could you use FetchContent instead of ExternalProject?
There was a problem hiding this comment.
Sure. Will refactor .
| HAVE_ALT | ||
| TRUE | ||
| REQUIRED_VERSION | ||
| "1.0.2" |
There was a problem hiding this comment.
Can we require 0.9.6 or later for Ubuntu 22.04?
There was a problem hiding this comment.
Specifically for ubuntu ? or do you want me to have the distribution bundled minimum for other distros as well? fedora has 1.0 and likewise.
There was a problem hiding this comment.
No. I think that Ubuntu 22.04 ships the oldest uriparser in our supported (CI checked) platforms. If we don't require 1.0.2 or later features, I want to accept older versions in supported platforms such as Ubuntu 22.04:
| "1.0.2" | |
| "0.9.6" |
If we accept older versions, we can reduce our CI time because we don't need to build bundled uriparser.
There was a problem hiding this comment.
I will update with >= 0.9.6
|
|
||
| arrow_util_deps = [threads_dep] | ||
| uriparser_dep = dependency( | ||
| 'liburiparser', |
There was a problem hiding this comment.
Does this search both of CMake package and pkg-config package?
There was a problem hiding this comment.
added uriparser as well so we can check for both the names
|
@github-actions crossbow submit -g r |
|
Revision: c885134 Submitted crossbow builds: ursacomputing/crossbow @ actions-f0e03a3a56 |
|
Could you check CI results and update the PR description? |
raulcd
left a comment
There was a problem hiding this comment.
@imtherealnaska could you rebase main, several of the CI failures are old and have been fixed on main. I want to assess the state of the PR as we would prefer to merge this before the 26.0.0 release and the feature freeze is expected for tomorrow.
|
The lint failure requires fixing, see the diff on the job failure: diff --git a/cpp/cmake_modules/FinduriparserAlt.cmake b/cpp/cmake_modules/FinduriparserAlt.cmake
index 7ef10f1841..0dd810d4fc 100644
--- a/cpp/cmake_modules/FinduriparserAlt.cmake
+++ b/cpp/cmake_modules/FinduriparserAlt.cmake
@@ -53,9 +53,10 @@ if(uriparser_PC_FOUND)
set(uriparserAlt_VERSION "${uriparser_PC_VERSION}")
endif()
-find_package_handle_standard_args(uriparserAlt
- REQUIRED_VARS uriparser_PC_FOUND
- VERSION_VAR uriparserAlt_VERSION)
+find_package_handle_standard_args(
+ uriparserAlt
+ REQUIRED_VARS uriparser_PC_FOUND
+ VERSION_VAR uriparserAlt_VERSION) |
…u and distro bundled version
9b47937 to
525b820
Compare
I had some more changes for Dockerfiles. I have tested them . MinIO failures should be fixed from rebase. |
|
of the 3 jobs failing 2 are MINGW at parquet-reader-test and are fixed by this PR . Conda forge is not sure |
|
I think the conda job failure is also because of simdjson 5.0.1 bug , but the env change in this PR forces a fresh solve so it's exposed i think . Should I pin simdjson in ci/conda_env_cpp.txt ? (on a different PR ? ) |
|
@github-actions crossbow submit -g r |
|
Revision: 525b820 Submitted crossbow builds: ursacomputing/crossbow @ actions-a46fa27440 |
raulcd
left a comment
There was a problem hiding this comment.
Thanks so much @imtherealnaska ! I am going to merge, I don't expect failures but if there are any failures on any of the extended CI checks we can fix them as part of the release preparation.
Rationale for this change
The vendored uriparser in
cpp/src/arrow/vendored/uriparser/is 0.9.3, which is old and misses upstream security fixes. Instead of refreshing the vendored copy, this makes uriparser an external dependency like our other third-party dependencies.What changes are included in this PR?
uriparsertoARROW_THIRDPARTY_DEPENDENCIESbuild_uriparser()that builds uriparser 1.0.2 withFetchContentforuriparser_SOURCE=BUNDLED(static; docs, tests, tools andwchar_tsupport are disabled). The built library is bundled intoarrow_bundled_dependencies.uriparser_SOURCE=SYSTEM. All uriparser APIs used by Arrow are available in 0.9.6, so supported platforms such as Ubuntu 22.04 can use their distribution package.cmake_modules/FinduriparserAlt.cmake. It tries uriparser's CMake package configuration first and falls back to pkg-config (liburiparser.pc), because Debian and Ubuntu'sliburiparser-devprovides only the pkg-config file.uriparser::uriparsertoARROW_STATIC_INSTALL_INTERFACE_LIBSforuriparser_SOURCE=SYSTEM, so that static library users link uriparser.subprojects/uriparser.wrapwith the CMake subproject support.cpp/src/arrow/vendored/uriparser/and its section inLICENSE.txt.Brewfile, vcpkg manifests, MSYS2 setup and Linux packages build environments.-Duriparser_SOURCEinci/scripts/cpp_build.sh.docs/source/developers/cpp/building.rst.NOTICE : The floor is 0.9.6 rather than 1.0.2, so supported platforms such as Ubuntu 22.04 can use their distro package and CI does not have to build uriparser from source.
Are these changes tested?
All existing tests pass.
Are there any user-facing changes?
Now Arrow requires uriparser or builds it . No user facing changes as such because no API changes .
This PR contains a "Critical Fix". It has security fixes that has gone in uriparser.
AI Usage: