Skip to content

feat(import): add support for multiple hbase snapshot imports#4600

Open
tianlei2 wants to merge 6 commits intogoogleapis:mainfrom
tianlei2:dataflow-import
Open

feat(import): add support for multiple hbase snapshot imports#4600
tianlei2 wants to merge 6 commits intogoogleapis:mainfrom
tianlei2:dataflow-import

Conversation

@tianlei2
Copy link
Copy Markdown

@tianlei2 tianlei2 commented Apr 28, 2026

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

b/429250716

This is the first PR that incorporates changes from https://github.com/jhambleton/java-bigtable-hbase/commits/dataflow-v2-v2.15.6 and some fixes to make it pass the tests.

  • Fixed Test Isolation Issues
    SnapshotUtilsTest.testGetHbaseConfiguration was failing because the static configuration field SnapshotUtils.hbaseConfiguration cached state between test cases, leaking stale data into subsequent tests.
    Solution: Added a @before setup method to reset the static field to null via reflection before every test run.
  • Fixed Timestamp Formatting Tests
    SnapshotUtilsTest.testAppendCurrentTimestamp was throwing a NumberFormatException because the return value contained a UUID suffix (timestamp-UUID), but the test attempted to parse the entire string directly as a Long.
    Solution: Updated the test to split the string using the "-" character to extract and correctly parse just the timestamp prefix.
  • Resolved Classpath and SPI Conflicts (dnsjava)
    Integration tests failed on Java 8 and 11 in Kokoro because of unshaded transitive dependency conflicts (com.google.protobuf.LiteralByteString NoClassDefFoundError).
    Solution: Reverted back to the shaded hbase-shaded-mapreduce dependency, ensuring proper compatibility across all Java versions.
  • Uncommented and Fixed Tests in ImportJobFromHbaseSnapshotTest
    Several useful unit tests were commented out in ImportJobFromHbaseSnapshotTest because mockito-core lacked the ability to mock static methods.
    Solution:
    Switched from mockito-core to mockito-inline in the pom.xml to allow static mocking.
    Uncommented the code and restored the original formatting to prevent any lint errors, enabling JUnit to verify correct configuration parsing.

@tianlei2 tianlei2 requested a review from a team as a code owner April 28, 2026 18:41
@product-auto-label product-auto-label Bot added size: xl Pull request size is extra large. api: bigtable Issues related to the googleapis/java-bigtable-hbase API. labels Apr 28, 2026
@tianlei2 tianlei2 force-pushed the dataflow-import branch 3 times, most recently from f5324bd to f8b5932 Compare April 28, 2026 19:22
@tianlei2 tianlei2 changed the title Dataflow import feat(import): add support for multiple hbase snapshot imports Apr 28, 2026
@tianlei2 tianlei2 force-pushed the dataflow-import branch 3 times, most recently from 13f65dc to a3a6c7f Compare April 28, 2026 20:17
@tianlei2 tianlei2 marked this pull request as draft April 28, 2026 20:42
@tianlei2 tianlei2 marked this pull request as ready for review April 28, 2026 22:41
@tianlei2 tianlei2 marked this pull request as draft April 28, 2026 22:42
@tianlei2 tianlei2 added the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 29, 2026
@tianlei2 tianlei2 self-assigned this Apr 29, 2026
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Apr 29, 2026
@tianlei2 tianlei2 force-pushed the dataflow-import branch 4 times, most recently from 12f1c11 to d511a61 Compare April 29, 2026 19:46
@tianlei2 tianlei2 requested a review from vermas2012 April 29, 2026 20:27
@googleapis googleapis deleted a comment from google-cla Bot Apr 29, 2026
@tianlei2 tianlei2 marked this pull request as ready for review April 29, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigtable Issues related to the googleapis/java-bigtable-hbase API. size: xl Pull request size is extra large.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants