[WIP] chore(bigframes): add specific build script for doctest to restrict execution#16711
[WIP] chore(bigframes): add specific build script for doctest to restrict execution#16711chalmerlowe wants to merge 6 commits intomainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new presubmit configuration and a bash script, run_doctest.sh, to automate doctests for the bigframes package. The script detects changes using git diff and executes nox sessions accordingly. The review feedback identifies several critical improvements for the bash script: correcting the nox command syntax to handle multiple sessions properly, adding necessary quoting to handle file paths with spaces, and implementing checks for Kokoro environment variables to prevent syntax errors in non-pull request builds.
| export NOX_SESSION="cleanup doctest" | ||
|
|
||
| cd "${package_path}" | ||
| python3 -m nox -s "${NOX_SESSION}" |
There was a problem hiding this comment.
The NOX_SESSION variable contains multiple session names separated by a space. When passed to nox -s inside double quotes, Nox treats the entire string as a single session name (e.g., "cleanup doctest"), which will fail if no such session exists. Additionally, NOX_SESSION is not a standard environment variable recognized by Nox; sessions should be specified via command-line arguments.
| export NOX_SESSION="cleanup doctest" | |
| cd "${package_path}" | |
| python3 -m nox -s "${NOX_SESSION}" | |
| cd "${package_path}" | |
| python3 -m nox -s cleanup -s doctest |
| # Assume we are running from the repo root or we need to find it. | ||
| # If this script is in packages/bigframes/scripts/run_doctest.sh, | ||
| # then repo root is 3 levels up. | ||
| export PROJECT_ROOT=$(realpath $(dirname "${BASH_SOURCE[0]}")/../../..) |
There was a problem hiding this comment.
| cd "$PROJECT_ROOT" | ||
|
|
||
| # This is needed in order for `git diff` to succeed | ||
| git config --global --add safe.directory $(realpath .) |
| echo "checking changes with 'git diff ${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT} -- ${files_to_check}'" | ||
| set +e | ||
| package_modified=$(git diff "${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT}" -- ${files_to_check} | wc -l) | ||
| set -e |
There was a problem hiding this comment.
The git diff command relies on environment variables that are typically only available in pull request builds. In other contexts (like continuous builds), these variables will be empty, leading to a git syntax error. While set +e prevents the script from terminating, it's cleaner to check for the presence of these variables. Additionally, the file path should be quoted.
| echo "checking changes with 'git diff ${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT} -- ${files_to_check}'" | |
| set +e | |
| package_modified=$(git diff "${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT}" -- ${files_to_check} | wc -l) | |
| set -e | |
| if [[ -n "${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}" && -n "${KOKORO_GITHUB_PULL_REQUEST_COMMIT}" ]]; then | |
| echo "checking changes with 'git diff ${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT} -- ${files_to_check}'" | |
| set +e | |
| package_modified=$(git diff "${KOKORO_GITHUB_PULL_REQUEST_TARGET_BRANCH}...${KOKORO_GITHUB_PULL_REQUEST_COMMIT}" -- "${files_to_check}" | wc -l) | |
| set -e | |
| else | |
| package_modified=0 | |
| fi |
|
Switching to draft until checks are green |
I just noticed that the test is skipped in the output
nox > Running session doctest
nox > Creating virtual environment (virtualenv) using python3.12 in .nox/doctest
nox > Session doctest skipped: Temporary skip to enable a PR merge. Remove skip as part of closing https://github.com/googleapis/google-cloud-python/issues/16489.
cleanup
…in run_doctest.sh
Warning
This is a work in progress. Do not review until invited.
Creates a build script:
packages/bigframes/scripts/run_doctest.sh. This script checks ifbigframeshas any changes (or if it's acontinuousbuild) before runnin' thecleanupdoctestsessions. This will prevent it from running when other packages are modified and failing the build.Updates the Kokoro config:
.kokoro/presubmit/presubmit-doctest-bigframes.cfgto setTRAMPOLINE_BUILD_FILEto point to the new script.