Skip to content

fix(deps-dev): Ensure user-scoped pipx installs and reliable version checks for pipx packages. #1320

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open

fix(deps-dev): Ensure user-scoped pipx installs and reliable version checks for pipx packages. #1320

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
e6817d8
remove sudo when installing pipx packages
Bill-hbrhbr Sep 23, 2025
1701f05
Apply suggestions from code review
Bill-hbrhbr Sep 23, 2025
896a9c4
line break for lint
Bill-hbrhbr Sep 23, 2025
12fc072
Preserve all env vars when running pipx install as user
Bill-hbrhbr Sep 23, 2025
24adfec
Add pipx-packages to trigger various build CI pipelines
Bill-hbrhbr Sep 23, 2025
84c01bf
Merge branch 'main' into sudo-install-lib
Bill-hbrhbr Sep 23, 2025
22ec79e
fix issue where the installed binary is not immediately available for…
Bill-hbrhbr Sep 25, 2025
59de3b0
Find installed pipx binary in the more reliable way
Bill-hbrhbr Sep 25, 2025
611596a
Merge branch 'main' into sudo-install-lib
Bill-hbrhbr Sep 25, 2025
ac5ec93
fix bug
Bill-hbrhbr Sep 25, 2025
d547a7c
Merge branch 'main' into sudo-install-lib
Bill-hbrhbr Sep 27, 2025
6bea8cf
Merge branch 'main' into sudo-install-lib
Bill-hbrhbr Sep 28, 2025
7523533
Merge branch 'main' into sudo-install-lib
Bill-hbrhbr Oct 1, 2025
4640164
Merge branch 'main' into sudo-install-lib
Bill-hbrhbr Oct 2, 2025
7a371fc
Merge branch 'main' into sudo-install-lib
Bill-hbrhbr Oct 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/clp-artifact-build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,24 +62,28 @@ jobs:
- ".github/actions/**"
- ".github/workflows/clp-artifact-build.yaml"
- "components/core/tools/scripts/lib_install/*.sh"
- "components/core/tools/scripts/lib_install/pipx-packages/**"
- "components/core/tools/docker-images/clp-env-base-centos-stream-9/**"
- "components/core/tools/scripts/lib_install/centos-stream-9/**"
manylinux_2_28_x86_64_image:
- ".github/actions/**"
- ".github/workflows/clp-artifact-build.yaml"
- "components/core/tools/scripts/lib_install/*.sh"
- "components/core/tools/scripts/lib_install/pipx-packages/**"
- "components/core/tools/docker-images/clp-env-base-manylinux_2_28-x86_64/**"
- "components/core/tools/scripts/lib_install/manylinux_2_28/**"
musllinux_1_2_x86_64_image:
- ".github/actions/**"
- ".github/workflows/clp-artifact-build.yaml"
- "components/core/tools/scripts/lib_install/*.sh"
- "components/core/tools/scripts/lib_install/pipx-packages/**"
- "components/core/tools/docker-images/clp-env-base-musllinux_1_2-x86_64/**"
- "components/core/tools/scripts/lib_install/musllinux_1_2/**"
ubuntu_jammy_image:
- ".github/actions/**"
- ".github/workflows/clp-artifact-build.yaml"
- "components/core/tools/scripts/lib_install/*.sh"
- "components/core/tools/scripts/lib_install/pipx-packages/**"
- "components/core/tools/docker-images/clp-env-base-ubuntu-jammy/**"
- "components/core/tools/scripts/lib_install/ubuntu-jammy/**"
clp:
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/clp-core-build-macos.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ on:
- "components/core/CMakeLists.txt"
- "components/core/src/**"
- "components/core/tests/**"
- "components/core/tools/scripts/lib_install/pipx-packages/**"
- "components/core/tools/scripts/lib_install/macos/**"
- "components/core/tools/scripts/deps-download/**"
- "components/core/tools/scripts/utils/build-and-run-unit-tests.py"
Expand All @@ -23,6 +24,7 @@ on:
- "components/core/CMakeLists.txt"
- "components/core/src/**"
- "components/core/tests/**"
- "components/core/tools/scripts/lib_install/pipx-packages/**"
- "components/core/tools/scripts/lib_install/macos/**"
- "components/core/tools/scripts/deps-download/**"
- "components/core/tools/scripts/utils/build-and-run-unit-tests.py"
Expand Down
7 changes: 7 additions & 0 deletions components/core/tools/scripts/lib_install/pipx-packages/install-all.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,13 @@ set -o errexit
set -o nounset
set -o pipefail

is_sudo_from_non_root=$(( EUID == 0 && ${SUDO_UID:-0} != 0 ))
if (( is_sudo_from_non_root )); then
echo "Installing pipx packages to the user environment (sudo lifted)."
exec sudo --preserve-env --set-home --user="$SUDO_USER" \
/usr/bin/env bash "${BASH_SOURCE[0]}" "$@"
fi

script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"

"${script_dir}/install-cmake.sh"
Expand Down
17 changes: 12 additions & 5 deletions components/core/tools/scripts/lib_install/pipx-packages/install-cmake.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,23 @@ readonly required_version_min="${required_version_major_min}.${required_version_
readonly required_version_major_max=3
readonly required_version_major_max_plus_1=$((required_version_major_max + 1))

package_preinstalled=0
if ! command -v cmake >/dev/null 2>&1; then
cmake_bin="$(command -v cmake 2>/dev/null || true)"
if [ -n "${cmake_bin}" ]; then
package_preinstalled=0
echo "Preinstalled CMake found at: ${cmake_bin}"
else
package_preinstalled=1
# ystdlib requires CMake v3.23; ANTLR and yaml-cpp do not yet support CMake v4+
# (see https://github.com/y-scope/clp/issues/795).
pipx install --force "cmake>=${required_version_min},<${required_version_major_max_plus_1}"
pipx ensurepath
cmake_bin="${PIPX_BIN_DIR:-$HOME/.local/bin}/cmake"
echo "Pipx CMake installed at: ${cmake_bin}"
fi

installed_version=$(cmake -E capabilities | jq --raw-output ".version.string")
installed_version_major=$(cmake -E capabilities | jq --raw-output ".version.major")
installed_version_minor=$(cmake -E capabilities | jq --raw-output ".version.minor")
installed_version=$("${cmake_bin}" -E capabilities | jq --raw-output ".version.string")
installed_version_major=$("${cmake_bin}" -E capabilities | jq --raw-output ".version.major")
installed_version_minor=$("${cmake_bin}" -E capabilities | jq --raw-output ".version.minor")

# ystdlib requires CMake v3.23; ANTLR and yaml-cpp do not yet support CMake v4+
# (see https://github.com/y-scope/clp/issues/795).
Expand All @@ -47,3 +52,5 @@ if (("${installed_version_major}" < "${required_version_major_min}")) \

exit 1
fi

echo "CMake version ${installed_version} installed at ${cmake_bin} satisfies version requirements."
17 changes: 12 additions & 5 deletions components/core/tools/scripts/lib_install/pipx-packages/install-go-task.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,22 @@ fi
# We lock to version 3.44.0 to avoid https://github.com/y-scope/clp-ffi-js/issues/110
readonly required_version="3.44.0"

package_preinstalled=0
if ! command -v task >/dev/null 2>&1; then
go_task_bin="$(command -v task 2>/dev/null || true)"
if [ -n "${go_task_bin}" ]; then
package_preinstalled=0
echo "Preinstalled Task found at: ${go_task_bin}"
else
package_preinstalled=1
pipx install --force "go-task-bin==${required_version}"
pipx ensurepath
go_task_bin="${PIPX_BIN_DIR:-$HOME/.local/bin}/task"
echo "Pipx Task installed at: ${go_task_bin}"
fi

script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
task_version=$(task --silent --taskfile "${script_dir}/print-go-task-version.yaml")
if [[ "${task_version}" != "${required_version}" ]]; then
echo "Error: Task version ${task_version} is currently unsupported (must be" \
installed_version=$(${go_task_bin} --silent --taskfile "${script_dir}/print-go-task-version.yaml")
if [[ "${installed_version}" != "${required_version}" ]]; then
echo "Error: Task version ${installed_version} is currently unsupported (must be" \
Comment on lines +29 to +31
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Quote the resolved binary when invoking.

Prevents breakage if the path contains spaces.

-installed_version=$(${go_task_bin} --silent --taskfile "${script_dir}/print-go-task-version.yaml")
+installed_version=$("${go_task_bin}" --silent --taskfile "${script_dir}/print-go-task-version.yaml")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
installed_version=$(${go_task_bin} --silent --taskfile "${script_dir}/print-go-task-version.yaml")
if [[ "${installed_version}" != "${required_version}" ]]; then
echo "Error: Task version ${installed_version} is currently unsupported (must be" \
installed_version=$("${go_task_bin}" --silent --taskfile "${script_dir}/print-go-task-version.yaml")
if [[ "${installed_version}" != "${required_version}" ]]; then
echo "Error: Task version ${installed_version} is currently unsupported (must be"
🤖 Prompt for AI Agents 
In components/core/tools/scripts/lib_install/pipx-packages/install-go-task.sh
around lines 29 to 31, the command substitution invokes the resolved binary
unquoted which breaks if the path contains spaces; update the invocation to
quote the binary (replace ${go_task_bin} with "${go_task_bin}") in the command
substitution (and any other places you call the binary) so the resolved path
with spaces is treated as a single argument.

"${required_version})."

if ((0 == "${package_preinstalled}")); then
Expand All @@ -35,3 +40,5 @@ if [[ "${task_version}" != "${required_version}" ]]; then

exit 1
fi

echo "Task version ${installed_version} installed at ${go_task_bin} satisfies version requirements."
11 changes: 9 additions & 2 deletions components/core/tools/scripts/lib_install/pipx-packages/install-uv.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,16 @@ readonly required_version_major_min=0
readonly required_version_minor_min=8
readonly required_version_min="${required_version_major_min}.${required_version_minor_min}"

package_preinstalled=0
if ! command -v uv >/dev/null 2>&1; then
uv_bin="$(command -v uv 2>/dev/null || true)"
if [ -n "${uv_bin}" ]; then
package_preinstalled=0
echo "Preinstalled uv found at: ${uv_bin}"
else
package_preinstalled=1
pipx install --force "uv>=${required_version_min}"
pipx ensurepath
uv_bin="${PIPX_BIN_DIR:-$HOME/.local/bin}/uv"
echo "Pipx uv installed at: ${uv_bin}"
fi

installed_version=$(uv self version --output-format json | jq --raw-output ".version")
Expand All @@ -38,3 +43,5 @@ if (("${installd_version_major}" == "${required_version_major_min}" && \

exit 1
fi

echo "uv version ${installed_version} installed at ${uv_bin} satisfies version requirements."
Loading