Fix Bazel build errors for OSS environment#50
Open
czgdp1807 wants to merge 1 commit intogoogle:masterfrom
Open
Fix Bazel build errors for OSS environment#50czgdp1807 wants to merge 1 commit intogoogle:masterfrom
czgdp1807 wants to merge 1 commit intogoogle:masterfrom
Conversation
- Add py_proto_library macro for OSS compatibility - Replace gunit_main with googletest gtest_main - Comment out Google-internal dependencies and missing test files - Fix proto import to use workspace-relative path - Remove invalid third_party load statements Fixes build errors when running `bazel build //...` in OSS environment.
Contributor
Author
|
Here are the build logs, (s2t-env) czgdp1807@qgpu3:~/struct2tensor$ bazel build /...
INFO: Build option --action_env has changed, discarding analysis cache.
WARNING: /home/czgdp1807/.cache/bazel/_bazel_czgdp1807/38445803433c11c40285b6c944f811b3/external/local_config_tf/BUILD:20551:8: target 'libtensorflow_framework.so' is both a rule and a file; please choose another name for the rule
INFO: Analyzed 173 targets (199 packages loaded, 12848 targets configured).
INFO: Found 173 targets...
INFO: From Compiling snappy-sinksource.cc:
cc1plus: warning: command-line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
INFO: From Compiling snappy-stubs-internal.cc:
cc1plus: warning: command-line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
INFO: From Compiling snappy.cc:
cc1plus: warning: command-line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
INFO: From Compiling src/google/protobuf/compiler/rust/relative_path.cc [for tool]:
external/com_google_protobuf/src/google/protobuf/compiler/rust/relative_path.cc: In member function 'std::string google::protobuf::compiler::rust::RelativePath::Relative(const google::protobuf::compiler::rust::RelativePath&) const':
external/com_google_protobuf/src/google/protobuf/compiler/rust/relative_path.cc:66:21: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<std::basic_string_view<char> >::size_type' {aka 'long unsigned int'} [-Wsign-compare]
66 | for (int i = 0; i < current_segments.size(); ++i) {
| ~~^~~~~~~~~~~~~~~~~~~~~~~~~
INFO: From Compiling src/google/protobuf/arena.cc [for tool]:
external/com_google_protobuf/src/google/protobuf/arena.cc: In member function 'void* google::protobuf::internal::SerialArena::AllocateAlignedFallback(size_t)':
external/com_google_protobuf/src/google/protobuf/arena.cc:200:10: warning: 'ret' may be used uninitialized [-Wmaybe-uninitialized]
200 | return ret;
| ^~~
external/com_google_protobuf/src/google/protobuf/arena.cc:197:9: note: 'ret' was declared here
197 | void* ret;
| ^~~
INFO: From Compiling snappy-sinksource.cc [for tool]:
cc1plus: warning: command-line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
INFO: From Compiling src/google/protobuf/message_lite.cc [for tool]:
external/com_google_protobuf/src/google/protobuf/message_lite.cc: In member function 'int google::protobuf::MessageLite::GetCachedSize() const':
external/com_google_protobuf/src/google/protobuf/message_lite.cc:69:70: warning: 'int google::protobuf::MessageLite::ByteSize() const' is deprecated: Please use ByteSizeLong() instead [-Wdeprecated-declarations]
69 | if (PROTOBUF_PREDICT_FALSE(cached_size == nullptr)) return ByteSize();
| ~~~~~~~~^~
In file included from external/com_google_protobuf/src/google/protobuf/message_lite.cc:13:
bazel-out/k8-opt-exec-50AE0418/bin/external/com_google_protobuf/src/google/protobuf/_virtual_includes/protobuf_lite/google/protobuf/message_lite.h:454:59: note: declared here
454 | [[deprecated("Please use ByteSizeLong() instead")]] int ByteSize() const {
| ^~~~~~~~
INFO: From Compiling snappy-stubs-internal.cc [for tool]:
cc1plus: warning: command-line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
INFO: From Compiling snappy.cc [for tool]:
cc1plus: warning: command-line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
INFO: From Compiling src/google/protobuf/generated_message_tctable_lite.cc [for tool]:
In file included from bazel-out/k8-opt-exec-50AE0418/bin/external/com_google_protobuf/src/google/protobuf/_virtual_includes/protobuf_lite/google/protobuf/generated_message_tctable_decl.h:22,
from external/com_google_protobuf/src/google/protobuf/generated_message_tctable_lite.cc:22:
bazel-out/k8-opt-exec-50AE0418/bin/external/com_google_protobuf/src/google/protobuf/_virtual_includes/protobuf_lite/google/protobuf/parse_context.h:1125:1: warning: 'always_inline' function might not be inlinable [-Wattributes]
1125 | ParseContext::ParseLengthDelimitedInlined(const char* ptr, const Func& func) {
| ^~~~~~~~~~~~
bazel-out/k8-opt-exec-50AE0418/bin/external/com_google_protobuf/src/google/protobuf/_virtual_includes/protobuf_lite/google/protobuf/parse_context.h:1125:1: warning: 'always_inline' function might not be inlinable [-Wattributes]
external/com_google_protobuf/src/google/protobuf/generated_message_tctable_lite.cc:830:36: warning: 'always_inline' function might not be inlinable [-Wattributes]
830 | PROTOBUF_ALWAYS_INLINE const char* TcParser::FastVarintS1(
| ^~~~~~~~
external/com_google_protobuf/src/google/protobuf/generated_message_tctable_lite.cc:830:36: warning: 'always_inline' function might not be inlinable [-Wattributes]
external/com_google_protobuf/src/google/protobuf/generated_message_tctable_lite.cc:753:29: warning: 'always_inline' function might not be inlinable [-Wattributes]
753 | PROTOBUF_ALWAYS_INLINE bool EnumIsValidAux(int32_t val, uint16_t xform_val,
| ^~~~~~~~~~~~~~
external/com_google_protobuf/src/google/protobuf/generated_message_tctable_lite.cc:741:29: warning: 'always_inline' function might not be inlinable [-Wattributes]
741 | PROTOBUF_ALWAYS_INLINE void PrefetchEnumData(uint16_t xform_val,
| ^~~~~~~~~~~~~~~~
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/any_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/descriptor_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/compiler/plugin_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/duration_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/empty_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/api_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/field_mask_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/struct_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/source_context_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/type_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/timestamp_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/wrappers_pb2.py:
external/com_google_protobuf/.: warning: directory does not exist.
INFO: From Generating Descriptor Set proto_library @com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto:
tensorflow_metadata/proto/v0/metric.proto:23:1: warning: Import tensorflow_metadata/proto/v0/path.proto is unused.
tensorflow_metadata/proto/v0/problem_statement.proto:20:1: warning: Import google/protobuf/struct.proto is unused.
INFO: From Generating proto_library @com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto:
tensorflow_metadata/proto/v0/metric.proto:23:1: warning: Import tensorflow_metadata/proto/v0/path.proto is unused.
tensorflow_metadata/proto/v0/problem_statement.proto:20:1: warning: Import google/protobuf/struct.proto is unused.
INFO: From Compiling tsl/platform/status.cc [for tool]:
external/local_tsl/tsl/platform/status.cc: In function 'std::vector<tsl::StackFrame> tsl::errors::GetStackTrace(const absl::lts_20230802::Status&)':
external/local_tsl/tsl/platform/status.cc:146:23: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<std::__cxx11::basic_string<char> >::size_type' {aka 'long unsigned int'} [-Wsign-compare]
146 | for (int i = 0; i < split.size() / 3; ++i) {
| ~~^~~~~~~~~~~~~~~~~~
INFO: Elapsed time: 154.236s, Critical Path: 146.36s
INFO: 2151 processes: 503 internal, 1648 processwrapper-sandbox.
INFO: Build completed successfully, 2151 total actions |
Contributor
Author
|
@vkarampudi If you want anything to be uncommented please let me know. I will see what I can do. There are a few things which I concluded to be Google-internal. My reasoning may be wrong, so please let me know in that case. Thank you. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR addresses multiple build failures when running
bazel build //...in the open-source environment by removing Google-internal dependencies and fixing proto import paths.Changes Made
Proto Library Support:
py_proto_librarymacro tostruct2tensor.bzlfor OSS compatibility with protobuf 4.xtest_any.prototo use workspace-relative path (struct2tensor/test/any.proto)third_party/protobuf/bazelandtools/build_defs/proto/cpppathsTest Framework:
gunit_mainwith OSS-compatible@com_google_googletest//:gtest_mainin parquet testskernels/parquet/BUILD:parent_indices_builder_test.ccparquet_reader_test.ccparquet_reader_util_test.ccserialization_benchmarkthat depends on internal benchmark frameworkDependencies:
//testing/pybaseand//testing/pybase:parameterized//file/colossus/public:cns//third_party/py/cpuinfo,//third_party/py/psutil,//third_party/py/yaml//third_party/py/tensorflow_docs//third_party/py/absl:appInvestigation Process
Build error analysis: Bazel errors explicitly indicated missing packages:
Workspace verification: Checked
third_party/directory structure and confirmed these paths don't exist in the OSS repositoryDependency pattern recognition: Distinguished internal vs OSS dependencies:
//path/to/internal(workspace-relative paths that don't exist)@external_repo//path(external dependencies like@absl_py//,@com_google_protobuf//)Iterative testing: Confirmed by progressively commenting out suspected internal dependencies and verifying build progress
Testing
bazel build //...completes successfully in OSS build environmentImpact