HBASE-29691: Change TableName.META_TABLE_NAME from being a global static #7558
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Apache9
left a comment
Apache9
left a comment
There was a problem hiding this comment.
I do not think this is the correct way, at least you should not let clients config meta table name right? They should ask the meta table name from bootstrap node.
Theoretically, we can even support changing meta table name at runtime, clients and region servers can ask meta table name from master when they hit table not found exception when accessing meta.
And if we think this is a bit difficult to implement, and maybe there is no need to change meta table name after initialization, then maybe we can introduce a config to set meta table name when bootstraping a cluster, then we record this name in master local region or on HDFS directly. But anyway, clients and region servers still need to ask for the meta table name from master, not from configuration. But they just do not need to deal with meta table name change when running.
Thanks.
| if (instance == null) { | ||
| synchronized (MetaTableName.class) { | ||
| if (instance == null) { | ||
| instance = initializeHbaseMetaTableName(HBaseConfiguration.create()); |
There was a problem hiding this comment.
I do not think this is a good practise, as this is still static, what if a client wants to connect different read replica clusters? And it will difficult to write UTs when you want to set up two clusters with different meta table names...
There was a problem hiding this comment.
I do not think this is a good practise, as this is still static
I agree, this does not change the static referring.
what if a client wants to connect different read replica clusters?
Could clients use different configurations for each cluster being accessed?
Could be as simple as the hbase.id approach (a single file under the root dir containing the meta suffix generated during bootstrap)? Then all processes would load that file when initialising? |
| private static TableName initializeHbaseMetaTableName(Configuration conf) { | ||
| TableName metaTableName = | ||
| TableName.valueOf(NamespaceDescriptor.SYSTEM_NAMESPACE_NAME_STR, "meta"); | ||
| LOG.info("Meta table suffix value: {}", metaTableName); | ||
| return metaTableName; |
There was a problem hiding this comment.
Where is the conf being used here? And the suffix?
| if (instance == null) { | ||
| synchronized (MetaTableName.class) { | ||
| if (instance == null) { | ||
| instance = initializeHbaseMetaTableName(HBaseConfiguration.create()); |
There was a problem hiding this comment.
I do not think this is a good practise, as this is still static
I agree, this does not change the static referring.
what if a client wants to connect different read replica clusters?
Could clients use different configurations for each cluster being accessed?
|
Shall we add meta table name to ZooKeeper quorums will be distinct between replica clusters, so it could be a good place to store meta table name during the bootstrap process. We don't need to support changing it in runtime. For instance, in cluster bootstrap process:
wdyt? |
|
Let's not introduce new persistent data on zookeeper, we could store it on HDFS on in the master local region. Maybe we should introduce a shell command like 'add_read_replica_cluster xxx' and execute it at the primary cluster, the primary cluster will generate the meta name suffix file or record for read replica clusters. |
HDFS won't be sufficient, because it's shared between the clusters. Could be in master local region, but that doesn't contradict with making it accessible via |
We coud use different file names for different read replica clusters? |
So basically, we read from the config: and create a file on HDFS: ? |
Was thinking of this approach. But after considering, if we could use different master local region for different HBase read replicas, we could just store this value in master local region? In this way the primary cluster does not need to know the read replicas. Maybe this is better? |
We can certainly do that, but in that case I'm not sure how to generate meta table names for the replica clusters. They have to be unique at the same storage location and with files we would able to scan existing files for conflicts. By storing it in master local region I'm not sure how to do it, unless we use UUIDs as suffixes. Same issue arises with the directory name of Master local region of a replica cluster. Shall we use the configured replica name or the generated suffix? Actually this might be the right way:
Question: how would the replica cluster know about its suffix? |
Anyway you need something in the configuration to tell a cluster that you are a read replica cluster or not, so you can just add a configuration named replica_suffix? |
That's where we started. It's already there: I think we started the conversation about whether we should use this replica.id in the meta table and master region directory name or generate a random value for each replica? |
As long as the replica id must be unique, I think use it as master local region's suffix is enough? For meta table, I'm nature on whether to reuse this id too, or we could generate one and store it in master local region, seems the former one is enough but the latter is more flexible, and as said above, theoretically we could even change the meta table name. Thanks. |
…tic: org.apache.hadoop.hbase
…tic: org.apache.hadoop.hbase.client
…tic: org.apache.hadoop.hbase.master
…tic: org.apache.hadoop.hbase.regionserver
…tic: fix spotless
…tic: Add dynamic meta table name discovery via ConnectionRegistry
…tic: Rename references
…tic: Rename references 2
…tic: Add meta table name in master region
…tic: Format code part 1
This comment has been minimized.
This comment has been minimized.
…tic: Format code part 2
…tic: Format code part 3: spotless
This comment has been minimized.
This comment has been minimized.
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
…tic: Format code part 4
5 participants
Jira: https://issues.apache.org/jira/browse/HBASE-29691