Skip to content

[WIP] chore(bigframes): add specific build script for doctest to restrict execution#16711

Draft
chalmerlowe wants to merge 6 commits intomainfrom
feat-bigframes-doctest-control
Draft

[WIP] chore(bigframes): add specific build script for doctest to restrict execution#16711
chalmerlowe wants to merge 6 commits intomainfrom
feat-bigframes-doctest-control

Conversation

@chalmerlowe
Copy link
Copy Markdown
Contributor

@chalmerlowe chalmerlowe commented Apr 17, 2026

Warning

This is a work in progress. Do not review until invited.

Creates a build script: packages/bigframes/scripts/run_doctest.sh. This script checks if bigframes has any changes (or if it's a continuous build) before runnin' the cleanup doctest sessions. This will prevent it from running when other packages are modified and failing the build.

Updates the Kokoro config: .kokoro/presubmit/presubmit-doctest-bigframes.cfg to set TRAMPOLINE_BUILD_FILE to point to the new script.

@chalmerlowe chalmerlowe requested review from a team as code owners April 17, 2026 15:30
@chalmerlowe chalmerlowe requested review from GarrettWu and removed request for a team April 17, 2026 15:30
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new presubmit configuration and a bash script, run_doctest.sh, to automate doctests for the bigframes package. The script detects changes using git diff and executes nox sessions accordingly. The review feedback identifies several critical improvements for the bash script: correcting the nox command syntax to handle multiple sessions properly, adding necessary quoting to handle file paths with spaces, and implementing checks for Kokoro environment variables to prevent syntax errors in non-pull request builds.

Comment on lines +36 to +39
export NOX_SESSION="cleanup doctest"

cd "${package_path}"
python3 -m nox -s "${NOX_SESSION}"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The NOX_SESSION variable contains multiple session names separated by a space. When passed to nox -s inside double quotes, Nox treats the entire string as a single session name (e.g., "cleanup doctest"), which will fail if no such session exists. Additionally, NOX_SESSION is not a standard environment variable recognized by Nox; sessions should be specified via command-line arguments.

Suggested change
export NOX_SESSION="cleanup doctest"
cd "${package_path}"
python3 -m nox -s "${NOX_SESSION}"
cd "${package_path}"
python3 -m nox -s cleanup -s doctest

# Assume we are running from the repo root or we need to find it.
# If this script is in packages/bigframes/scripts/run_doctest.sh,
# then repo root is 3 levels up.
export PROJECT_ROOT=$(realpath $(dirname "${BASH_SOURCE[0]}")/../../..)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The argument to realpath should be quoted to correctly handle directory paths that may contain spaces.

Suggested change
export PROJECT_ROOT=$(realpath $(dirname "${BASH_SOURCE[0]}")/../../..)
export PROJECT_ROOT=$(realpath "$(dirname "${BASH_SOURCE[0]}")/../../..")

cd "$PROJECT_ROOT"

# This is needed in order for `git diff` to succeed
git config --global --add safe.directory $(realpath .)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The output of realpath . should be quoted to ensure the command works correctly if the current directory path contains spaces.

Suggested change
git config --global --add safe.directory $(realpath .)
git config --global --add safe.directory "$(realpath .)"

Comment on lines +25 to +28
echo "checking changes with 'git diff ${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT} -- ${files_to_check}'"
set +e
package_modified=$(git diff "${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT}" -- ${files_to_check} | wc -l)
set -e
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The git diff command relies on environment variables that are typically only available in pull request builds. In other contexts (like continuous builds), these variables will be empty, leading to a git syntax error. While set +e prevents the script from terminating, it's cleaner to check for the presence of these variables. Additionally, the file path should be quoted.

Suggested change
echo "checking changes with 'git diff ${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT} -- ${files_to_check}'"
set +e
package_modified=$(git diff "${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT}" -- ${files_to_check} | wc -l)
set -e
if [[ -n "${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}" && -n "${KOKORO_GITHUB_PULL_REQUEST_COMMIT}" ]]; then
echo "checking changes with 'git diff ${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT} -- ${files_to_check}'"
set +e
package_modified=$(git diff "${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT}" -- "${files_to_check}" | wc -l)
set -e
else
package_modified=0
fi

@parthea
Copy link
Copy Markdown
Contributor

parthea commented Apr 17, 2026

Switching to draft until checks are green

------------------------------------------------------------
Running doctest for: bigframes
------------------------------------------------------------
nox > Error while collecting sessions.
nox > Sessions not found: cleanup doctest

@parthea parthea marked this pull request as draft April 17, 2026 15:38
parthea
parthea previously approved these changes Apr 17, 2026
@parthea parthea dismissed their stale review April 17, 2026 19:43

I just noticed that the test is skipped in the output

nox > Running session doctest
nox > Creating virtual environment (virtualenv) using python3.12 in .nox/doctest
nox > Session doctest skipped: Temporary skip to enable a PR merge. Remove skip as part of closing https://github.com/googleapis/google-cloud-python/issues/16489.
cleanup
@chalmerlowe chalmerlowe self-assigned this Apr 17, 2026
@chalmerlowe chalmerlowe changed the title chore(bigframes): add specific build script for doctest to restrict execution [WIP] chore(bigframes): add specific build script for doctest to restrict execution Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants