[==========] Running 4 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 4 tests from TokenSignerITest
[ RUN ] TokenSignerITest.TskAtLeaderMaster
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260926 13:42:03.357801 392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:41999,127.0.98.61:34681,127.0.98.60:34013
I20260926 13:42:03.359169 392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 13:42:03.360169 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:03.368599 398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.368700 400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.368700 397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.368866 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.369408 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.369513 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.369565 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123369562 us; error 0 us; skew 500 ppm
I20260926 13:42:03.371817 392 webserver.cc:460] Webserver started at http://127.0.98.62:44435/ using document root <none> and password file <none>
I20260926 13:42:03.372362 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.372430 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.372680 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.374879 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-0-root/instance:
uuid: "bb411a5ff40241fc8f976288268e0ad7"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.379033 392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 13:42:03.381382 405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.382745 392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 13:42:03.382856 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-0-root
uuid: "bb411a5ff40241fc8f976288268e0ad7"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.382951 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.404505 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.405184 392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 13:42:03.405340 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.412226 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:41999
I20260926 13:42:03.412222 457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:41999 every 8 connection(s)
I20260926 13:42:03.413409 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:03.414979 460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.415078 461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.415181 463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.415385 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.415546 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.415582 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.415602 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123415601 us; error 0 us; skew 500 ppm
I20260926 13:42:03.416105 458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:42:03.416558 392 webserver.cc:460] Webserver started at http://127.0.98.61:44857/ using document root <none> and password file <none>
I20260926 13:42:03.416720 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.416774 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.416841 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.417274 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-1-root/instance:
uuid: "10a295f691c04927b2a3e3f8998173f4"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.418912 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:42:03.419910 468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.420161 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.420238 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-1-root
uuid: "10a295f691c04927b2a3e3f8998173f4"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.420307 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.422801 458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.431600 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.431960 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.435484 458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:03.436240 408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.98.61:34681: connect: Connection refused (error 111)
W20260926 13:42:03.437137 458 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.98.61:34681: Network error: Client connection negotiation failed: client connection to 127.0.98.61:34681: connect: Connection refused (error 111)
I20260926 13:42:03.437314 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.61:34681
I20260926 13:42:03.437340 523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.61:34681 every 8 connection(s)
I20260926 13:42:03.437922 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:42:03.438095 524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:42:03.439225 524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:03.439352 526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.439450 527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.439833 529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.439909 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.440120 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.440182 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.440203 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123440203 us; error 0 us; skew 500 ppm
I20260926 13:42:03.441008 392 webserver.cc:460] Webserver started at http://127.0.98.60:44823/ using document root <none> and password file <none>
I20260926 13:42:03.441129 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.441176 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.441237 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.441608 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-2-root/instance:
uuid: "1a73feea9e0646ed9fd18e8c48205ab9"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.443244 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:42:03.444033 524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.444211 535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.444453 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.444522 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-2-root
uuid: "1a73feea9e0646ed9fd18e8c48205ab9"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.444634 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.448508 524 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:03.449635 524 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.98.60:34013: Network error: Client connection negotiation failed: client connection to 127.0.98.60:34013: connect: Connection refused (error 111)
I20260926 13:42:03.475080 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.475495 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.480020 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.60:34013
I20260926 13:42:03.480079 589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.60:34013 every 8 connection(s)
I20260926 13:42:03.480439 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 13:42:03.480914 590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:42:03.482038 590 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.486348 590 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.486368 458 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } attempt: 1
I20260926 13:42:03.490489 590 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.490535 458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.497121 590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9: Bootstrap starting.
I20260926 13:42:03.497121 458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7: Bootstrap starting.
I20260926 13:42:03.499747 458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:03.499748 590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:03.500591 590 log.cc:824] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9: Log is configured to *not* fsync() on all Append() calls
I20260926 13:42:03.501788 524 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } attempt: 1
I20260926 13:42:03.502372 458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7: No bootstrap required, opened a new log
I20260926 13:42:03.502372 590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9: No bootstrap required, opened a new log
I20260926 13:42:03.506861 590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } }
I20260926 13:42:03.506871 458 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } }
I20260926 13:42:03.507109 590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:03.507125 524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4: Bootstrap starting.
I20260926 13:42:03.507109 458 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:03.507215 590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a73feea9e0646ed9fd18e8c48205ab9, State: Initialized, Role: FOLLOWER
I20260926 13:42:03.507228 458 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb411a5ff40241fc8f976288268e0ad7, State: Initialized, Role: FOLLOWER
I20260926 13:42:03.507797 458 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } }
I20260926 13:42:03.507797 590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } }
I20260926 13:42:03.508009 524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:03.508318 598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } } }
I20260926 13:42:03.508440 598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:03.508601 599 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } } }
I20260926 13:42:03.508728 599 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:03.509234 524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4: No bootstrap required, opened a new log
I20260926 13:42:03.509338 458 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:03.509338 590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:03.509606 524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } }
I20260926 13:42:03.509699 524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:03.509728 524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a295f691c04927b2a3e3f8998173f4, State: Initialized, Role: FOLLOWER
I20260926 13:42:03.509840 524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } }
I20260926 13:42:03.510098 601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } } }
I20260926 13:42:03.510182 601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:03.510253 524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:03.511520 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260926 13:42:03.512020 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260926 13:42:03.512588 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:03.514523 624 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:03.514616 624 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 13:42:03.514520 631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:03.514714 631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 13:42:03.514523 634 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:03.514782 634 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:42:03.516822 598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:42:03.516965 598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } }
W20260926 13:42:03.517849 637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.517932 639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.518042 598 leader_election.cc:288] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb411a5ff40241fc8f976288268e0ad7 (127.0.98.62:41999), 10a295f691c04927b2a3e3f8998173f4 (127.0.98.61:34681)
W20260926 13:42:03.517894 636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.518477 496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a295f691c04927b2a3e3f8998173f4" is_pre_election: true
I20260926 13:42:03.518479 432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb411a5ff40241fc8f976288268e0ad7" is_pre_election: true
I20260926 13:42:03.518731 432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a73feea9e0646ed9fd18e8c48205ab9 in term 0.
I20260926 13:42:03.518731 496 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a73feea9e0646ed9fd18e8c48205ab9 in term 0.
I20260926 13:42:03.519104 539 leader_election.cc:302] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a73feea9e0646ed9fd18e8c48205ab9, bb411a5ff40241fc8f976288268e0ad7; no voters:
I20260926 13:42:03.519320 598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:42:03.519395 598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:42:03.519441 598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:03.520781 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.520865 598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } }
I20260926 13:42:03.520970 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.521027 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.521055 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123521055 us; error 0 us; skew 500 ppm
I20260926 13:42:03.521174 598 leader_election.cc:288] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [CANDIDATE]: Term 1 election: Requested vote from peers bb411a5ff40241fc8f976288268e0ad7 (127.0.98.62:41999), 10a295f691c04927b2a3e3f8998173f4 (127.0.98.61:34681)
I20260926 13:42:03.521531 496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a295f691c04927b2a3e3f8998173f4"
I20260926 13:42:03.521636 496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:03.522356 496 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a73feea9e0646ed9fd18e8c48205ab9 in term 1.
I20260926 13:42:03.522532 432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb411a5ff40241fc8f976288268e0ad7"
I20260926 13:42:03.522589 539 leader_election.cc:302] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10a295f691c04927b2a3e3f8998173f4, 1a73feea9e0646ed9fd18e8c48205ab9; no voters:
I20260926 13:42:03.522626 432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:03.522748 598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:42:03.522874 598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 1 LEADER]: Becoming Leader. State: Replica: 1a73feea9e0646ed9fd18e8c48205ab9, State: Running, Role: LEADER
I20260926 13:42:03.523123 392 webserver.cc:460] Webserver started at http://127.0.98.1:32875/ using document root <none> and password file <none>
I20260926 13:42:03.523247 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.523207 598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } }
I20260926 13:42:03.523286 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.523345 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.523702 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-0-root/instance:
uuid: "d4ad7628f7d44c8097fac3f1e2c0e09a"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.523715 432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a73feea9e0646ed9fd18e8c48205ab9 in term 1.
I20260926 13:42:03.525398 642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a73feea9e0646ed9fd18e8c48205ab9. Latest consensus state: current_term: 1 leader_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } } }
I20260926 13:42:03.525496 642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:03.525833 646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:42:03.526546 646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:42:03.528925 392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260926 13:42:03.530006 648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.530238 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.530306 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-0-root
uuid: "d4ad7628f7d44c8097fac3f1e2c0e09a"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.530360 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.531383 432 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [term 1 FOLLOWER]: Refusing update from remote peer 1a73feea9e0646ed9fd18e8c48205ab9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:42:03.531383 496 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [term 1 FOLLOWER]: Refusing update from remote peer 1a73feea9e0646ed9fd18e8c48205ab9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:42:03.531909 598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:42:03.532047 642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:42:03.533859 599 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a73feea9e0646ed9fd18e8c48205ab9. Latest consensus state: current_term: 1 leader_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } } }
I20260926 13:42:03.533870 601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a73feea9e0646ed9fd18e8c48205ab9. Latest consensus state: current_term: 1 leader_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } } }
I20260926 13:42:03.533979 599 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:03.533988 601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:03.535482 642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } } }
I20260926 13:42:03.535569 642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:03.535641 599 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } } }
I20260926 13:42:03.535692 599 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:03.536010 647 mvcc.cc:205] Tried to move back new op lower bound from 7333601785977917440 to 7333601785954992128. Current Snapshot: MvccSnapshot[applied={T|T < 7333601785977917440}]
I20260926 13:42:03.537940 598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } } }
I20260926 13:42:03.538036 598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:03.538357 646 catalog_manager.cc:1347] Generated new cluster ID: e07815fadd594c17bd3eb138b0c21a70
I20260926 13:42:03.538441 646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:42:03.538774 601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb411a5ff40241fc8f976288268e0ad7" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 41999 } } peers { permanent_uuid: "10a295f691c04927b2a3e3f8998173f4" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 34681 } } peers { permanent_uuid: "1a73feea9e0646ed9fd18e8c48205ab9" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 34013 } } }
I20260926 13:42:03.538874 601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:03.546168 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.546612 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.547049 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:03.547994 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:03.548097 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.548202 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:03.548265 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.552205 646 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:42:03.553537 646 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:42:03.554906 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:36909
I20260926 13:42:03.554947 719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:36909 every 8 connection(s)
I20260926 13:42:03.556154 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:03.560024 726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.560290 725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.560442 728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.561084 720 heartbeater.cc:347] Connected to a master server at 127.0.98.62:41999
I20260926 13:42:03.561199 720 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.561691 720 heartbeater.cc:511] Master 127.0.98.62:41999 requested a full tablet report, sending...
I20260926 13:42:03.561993 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.562199 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.562253 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.562309 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123562307 us; error 0 us; skew 500 ppm
I20260926 13:42:03.563378 392 webserver.cc:460] Webserver started at http://127.0.98.2:34969/ using document root <none> and password file <none>
I20260926 13:42:03.563546 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.563544 420 ts_manager.cc:194] Registered new tserver with Master: d4ad7628f7d44c8097fac3f1e2c0e09a (127.0.98.1:36909)
I20260926 13:42:03.563637 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.564512 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.564987 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-1-root/instance:
uuid: "92cc1502311840b9a4ad449b6763ae95"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.566887 646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9: Generated new TSK 0
I20260926 13:42:03.567694 646 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:42:03.569659 722 heartbeater.cc:347] Connected to a master server at 127.0.98.61:34681
I20260926 13:42:03.569742 722 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.569940 722 heartbeater.cc:511] Master 127.0.98.61:34681 requested a full tablet report, sending...
I20260926 13:42:03.570569 479 ts_manager.cc:194] Registered new tserver with Master: d4ad7628f7d44c8097fac3f1e2c0e09a (127.0.98.1:36909)
I20260926 13:42:03.571261 721 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34013
I20260926 13:42:03.571337 721 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.571480 721 heartbeater.cc:511] Master 127.0.98.60:34013 requested a full tablet report, sending...
I20260926 13:42:03.571957 554 ts_manager.cc:194] Registered new tserver with Master: d4ad7628f7d44c8097fac3f1e2c0e09a (127.0.98.1:36909)
I20260926 13:42:03.572942 392 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.001s
I20260926 13:42:03.573351 554 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52390
I20260926 13:42:03.576772 735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.577014 392 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260926 13:42:03.577080 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-1-root
uuid: "92cc1502311840b9a4ad449b6763ae95"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.577145 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.595956 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.596383 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.596705 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:03.597232 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:03.597277 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.597334 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:03.597363 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.603449 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:33167
I20260926 13:42:03.603583 798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:33167 every 8 connection(s)
I20260926 13:42:03.604419 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:03.610198 809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.610392 799 heartbeater.cc:347] Connected to a master server at 127.0.98.62:41999
W20260926 13:42:03.610426 806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.610633 807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.610484 799 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.611028 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.611090 799 heartbeater.cc:511] Master 127.0.98.62:41999 requested a full tablet report, sending...
I20260926 13:42:03.611207 801 heartbeater.cc:347] Connected to a master server at 127.0.98.61:34681
I20260926 13:42:03.611277 801 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.611258 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.611311 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.611339 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123611339 us; error 0 us; skew 500 ppm
I20260926 13:42:03.611312 800 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34013
I20260926 13:42:03.611424 801 heartbeater.cc:511] Master 127.0.98.61:34681 requested a full tablet report, sending...
I20260926 13:42:03.611442 800 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.611615 800 heartbeater.cc:511] Master 127.0.98.60:34013 requested a full tablet report, sending...
I20260926 13:42:03.612182 392 webserver.cc:460] Webserver started at http://127.0.98.3:32837/ using document root <none> and password file <none>
I20260926 13:42:03.612365 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.612421 479 ts_manager.cc:194] Registered new tserver with Master: 92cc1502311840b9a4ad449b6763ae95 (127.0.98.2:33167)
I20260926 13:42:03.612434 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.612561 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.612656 420 ts_manager.cc:194] Registered new tserver with Master: 92cc1502311840b9a4ad449b6763ae95 (127.0.98.2:33167)
I20260926 13:42:03.612440 554 ts_manager.cc:194] Registered new tserver with Master: 92cc1502311840b9a4ad449b6763ae95 (127.0.98.2:33167)
I20260926 13:42:03.612988 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-2-root/instance:
uuid: "f00280408ef540a7b36949630c6a004c"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.613577 554 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52400
I20260926 13:42:03.614636 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:42:03.615484 814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.615679 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.615731 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-2-root
uuid: "f00280408ef540a7b36949630c6a004c"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.615797 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskAtLeaderMaster.1790430123331008-392-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.632731 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.633137 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.633419 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:03.633888 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:03.633924 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.633955 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:03.633981 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.639688 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.3:38981
I20260926 13:42:03.639782 877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.3:38981 every 8 connection(s)
I20260926 13:42:03.646042 878 heartbeater.cc:347] Connected to a master server at 127.0.98.62:41999
I20260926 13:42:03.646129 878 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.646363 878 heartbeater.cc:511] Master 127.0.98.62:41999 requested a full tablet report, sending...
I20260926 13:42:03.647008 420 ts_manager.cc:194] Registered new tserver with Master: f00280408ef540a7b36949630c6a004c (127.0.98.3:38981)
I20260926 13:42:03.648346 879 heartbeater.cc:347] Connected to a master server at 127.0.98.60:34013
I20260926 13:42:03.648432 879 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.648588 879 heartbeater.cc:511] Master 127.0.98.60:34013 requested a full tablet report, sending...
I20260926 13:42:03.648913 880 heartbeater.cc:347] Connected to a master server at 127.0.98.61:34681
I20260926 13:42:03.649034 880 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.649071 554 ts_manager.cc:194] Registered new tserver with Master: f00280408ef540a7b36949630c6a004c (127.0.98.3:38981)
I20260926 13:42:03.649230 880 heartbeater.cc:511] Master 127.0.98.61:34681 requested a full tablet report, sending...
I20260926 13:42:03.649657 554 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52406
I20260926 13:42:03.649818 479 ts_manager.cc:194] Registered new tserver with Master: f00280408ef540a7b36949630c6a004c (127.0.98.3:38981)
I20260926 13:42:03.650578 392 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.010343422s
I20260926 13:42:03.652094 392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260926 13:42:03.656581 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:03.669499 392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260926 13:42:03.673342 392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260926 13:42:03.676896 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:03.688750 392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260926 13:42:03.690382 392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260926 13:42:03.693733 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:03.705582 392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260926 13:42:03.707197 392 master.cc:537] Master@127.0.98.62:41999 shutting down...
I20260926 13:42:03.710562 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:42:03.710850 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:03.710937 392 tablet_replica.cc:330] T 00000000000000000000000000000000 P bb411a5ff40241fc8f976288268e0ad7: stopping tablet replica
I20260926 13:42:03.724220 392 master.cc:552] Master@127.0.98.62:41999 shutdown complete.
I20260926 13:42:03.727133 392 master.cc:537] Master@127.0.98.61:34681 shutting down...
I20260926 13:42:03.729561 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:42:03.729677 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:03.729743 392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 10a295f691c04927b2a3e3f8998173f4: stopping tablet replica
I20260926 13:42:03.742120 392 master.cc:552] Master@127.0.98.61:34681 shutdown complete.
I20260926 13:42:03.744360 392 master.cc:537] Master@127.0.98.60:34013 shutting down...
I20260926 13:42:03.747153 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:42:03.747437 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:03.747545 392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1a73feea9e0646ed9fd18e8c48205ab9: stopping tablet replica
I20260926 13:42:03.759786 392 master.cc:552] Master@127.0.98.60:34013 shutdown complete.
[ OK ] TokenSignerITest.TskAtLeaderMaster (408 ms)
[ RUN ] TokenSignerITest.TskClusterRestart
I20260926 13:42:03.765296 392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:33977,127.0.98.61:41179,127.0.98.60:44091
I20260926 13:42:03.765647 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:03.767060 886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.767177 889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.767208 887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.767316 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.767437 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.767468 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.767488 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123767487 us; error 0 us; skew 500 ppm
I20260926 13:42:03.768217 392 webserver.cc:460] Webserver started at http://127.0.98.62:42963/ using document root <none> and password file <none>
I20260926 13:42:03.768350 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.768393 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.768460 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.768888 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-0-root/instance:
uuid: "f2e5ebc231eb4a93b1961f79de763e12"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.770236 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.771013 894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.771205 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.771267 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-0-root
uuid: "f2e5ebc231eb4a93b1961f79de763e12"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.771332 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.806478 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.806805 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.810614 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:33977
I20260926 13:42:03.811661 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:42:03.813719 946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:33977 every 8 connection(s)
I20260926 13:42:03.813856 947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 13:42:03.814649 950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.814711 949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.814644 952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.814916 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.815064 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.815101 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.815115 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123815115 us; error 0 us; skew 500 ppm
I20260926 13:42:03.815337 947 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.815943 392 webserver.cc:460] Webserver started at http://127.0.98.61:37653/ using document root <none> and password file <none>
I20260926 13:42:03.816079 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.816121 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.816191 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.816509 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-1-root/instance:
uuid: "14358fff8b70466598979e4e937c5d8e"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.818138 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:42:03.819046 959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.819271 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:42:03.819329 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-1-root
uuid: "14358fff8b70466598979e4e937c5d8e"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.819389 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.820533 947 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:03.821586 947 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.98.61:41179: Network error: Client connection negotiation failed: client connection to 127.0.98.61:41179: connect: Connection refused (error 111)
I20260926 13:42:03.825141 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.825397 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.828984 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.61:41179
I20260926 13:42:03.829001 1012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.61:41179 every 8 connection(s)
I20260926 13:42:03.829535 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:42:03.829613 1013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:42:03.830734 1013 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:03.830981 1015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.831027 1016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.831240 1018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.831437 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.831591 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.831627 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.831646 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123831646 us; error 0 us; skew 500 ppm
I20260926 13:42:03.832332 392 webserver.cc:460] Webserver started at http://127.0.98.60:42349/ using document root <none> and password file <none>
I20260926 13:42:03.832463 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.832504 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.832563 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.832892 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-2-root/instance:
uuid: "b9f98437d057463d9c176d211c44ad79"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.834247 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.835029 1013 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.835160 1024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.835371 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.835439 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-2-root
uuid: "b9f98437d057463d9c176d211c44ad79"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.835505 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.839257 1013 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:03.840287 1013 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.98.60:44091: Network error: Client connection negotiation failed: client connection to 127.0.98.60:44091: connect: Connection refused (error 111)
I20260926 13:42:03.846565 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.846829 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.850426 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.60:44091
I20260926 13:42:03.850446 1078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.60:44091 every 8 connection(s)
I20260926 13:42:03.850754 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 13:42:03.851074 1079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:42:03.851999 1079 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.855566 1079 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.858866 1079 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.863839 1079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: Bootstrap starting.
I20260926 13:42:03.864811 1079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:03.865703 1079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: No bootstrap required, opened a new log
I20260926 13:42:03.866077 1079 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:03.866163 1079 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:03.866194 1079 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9f98437d057463d9c176d211c44ad79, State: Initialized, Role: FOLLOWER
I20260926 13:42:03.866313 1079 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:03.866580 1084 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:03.866670 1084 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:03.866714 1079 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 13:42:03.868551 1095 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:03.868630 1095 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:42:03.879729 947 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } attempt: 1
I20260926 13:42:03.883574 947 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:03.887840 947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: Bootstrap starting.
I20260926 13:42:03.888651 947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:03.889573 947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: No bootstrap required, opened a new log
I20260926 13:42:03.889942 947 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:03.890026 947 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:03.890056 947 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e5ebc231eb4a93b1961f79de763e12, State: Initialized, Role: FOLLOWER
I20260926 13:42:03.890168 947 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:03.890450 1097 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:03.890535 1097 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:03.890579 947 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:03.891945 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20260926 13:42:03.892354 1108 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:03.892408 1108 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:42:03.905432 1013 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } attempt: 1
I20260926 13:42:03.909695 1013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: Bootstrap starting.
I20260926 13:42:03.910459 1013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:03.911365 1013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: No bootstrap required, opened a new log
I20260926 13:42:03.911700 1013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:03.911782 1013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:03.911801 1013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14358fff8b70466598979e4e937c5d8e, State: Initialized, Role: FOLLOWER
I20260926 13:42:03.911890 1013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:03.912197 1110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:03.912259 1013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:03.912304 1110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:03.913656 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260926 13:42:03.914131 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:03.914255 1121 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:03.914310 1121 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 13:42:03.915791 1122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.915820 1125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.915974 1123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.915997 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.916179 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.916215 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.916235 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123916235 us; error 0 us; skew 500 ppm
I20260926 13:42:03.917088 392 webserver.cc:460] Webserver started at http://127.0.98.1:34593/ using document root <none> and password file <none>
I20260926 13:42:03.917218 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.917263 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.917327 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.917646 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-0-root/instance:
uuid: "ab856792f41e4bb09690d378cd45d833"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.918931 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.919754 1130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.919967 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.920028 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-0-root
uuid: "ab856792f41e4bb09690d378cd45d833"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.920087 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.936723 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.937034 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.937279 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:03.937695 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:03.937729 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.937768 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:03.937796 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.941969 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:42373
I20260926 13:42:03.942011 1193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:42373 every 8 connection(s)
I20260926 13:42:03.942793 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:03.944751 1204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.944957 1202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.945631 1201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.945767 392 server_base.cc:1034] running on GCE node
I20260926 13:42:03.946008 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.946058 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.946086 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123946086 us; error 0 us; skew 500 ppm
I20260926 13:42:03.947346 392 webserver.cc:460] Webserver started at http://127.0.98.2:37427/ using document root <none> and password file <none>
I20260926 13:42:03.947492 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.947542 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.947606 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.948017 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-1-root/instance:
uuid: "3e63508cf2d6474bb7272476a8b4de85"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.948153 1195 heartbeater.cc:347] Connected to a master server at 127.0.98.62:33977
I20260926 13:42:03.948238 1195 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.948405 1195 heartbeater.cc:511] Master 127.0.98.62:33977 requested a full tablet report, sending...
I20260926 13:42:03.948683 1196 heartbeater.cc:347] Connected to a master server at 127.0.98.61:41179
I20260926 13:42:03.948748 1196 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.948820 1194 heartbeater.cc:347] Connected to a master server at 127.0.98.60:44091
I20260926 13:42:03.948890 1194 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.948906 1196 heartbeater.cc:511] Master 127.0.98.61:41179 requested a full tablet report, sending...
I20260926 13:42:03.949036 1194 heartbeater.cc:511] Master 127.0.98.60:44091 requested a full tablet report, sending...
I20260926 13:42:03.949144 911 ts_manager.cc:194] Registered new tserver with Master: ab856792f41e4bb09690d378cd45d833 (127.0.98.1:42373)
I20260926 13:42:03.949492 977 ts_manager.cc:194] Registered new tserver with Master: ab856792f41e4bb09690d378cd45d833 (127.0.98.1:42373)
I20260926 13:42:03.949774 1043 ts_manager.cc:194] Registered new tserver with Master: ab856792f41e4bb09690d378cd45d833 (127.0.98.1:42373)
I20260926 13:42:03.949858 392 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:42:03.950732 1209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.950943 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.951000 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-1-root
uuid: "3e63508cf2d6474bb7272476a8b4de85"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.951057 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.958251 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.958557 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.958789 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:03.959174 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:03.959209 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.959246 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:03.959275 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.963418 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:33167
I20260926 13:42:03.963469 1272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:33167 every 8 connection(s)
I20260926 13:42:03.964257 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:03.966707 1281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:03.967458 1280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.967561 392 server_base.cc:1034] running on GCE node
W20260926 13:42:03.967687 1283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:03.967883 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:03.967921 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:03.967940 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430123967939 us; error 0 us; skew 500 ppm
I20260926 13:42:03.969132 392 webserver.cc:460] Webserver started at http://127.0.98.3:40777/ using document root <none> and password file <none>
I20260926 13:42:03.969286 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:03.969339 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:03.969405 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:03.969813 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-2-root/instance:
uuid: "e85d5327b7194133aef86e19d12c9767"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.969949 1275 heartbeater.cc:347] Connected to a master server at 127.0.98.61:41179
I20260926 13:42:03.970019 1274 heartbeater.cc:347] Connected to a master server at 127.0.98.62:33977
I20260926 13:42:03.970077 1274 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.970167 1273 heartbeater.cc:347] Connected to a master server at 127.0.98.60:44091
I20260926 13:42:03.970229 1273 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.970252 1274 heartbeater.cc:511] Master 127.0.98.62:33977 requested a full tablet report, sending...
I20260926 13:42:03.970350 1273 heartbeater.cc:511] Master 127.0.98.60:44091 requested a full tablet report, sending...
I20260926 13:42:03.970031 1275 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:03.970569 1275 heartbeater.cc:511] Master 127.0.98.61:41179 requested a full tablet report, sending...
I20260926 13:42:03.970949 1043 ts_manager.cc:194] Registered new tserver with Master: 3e63508cf2d6474bb7272476a8b4de85 (127.0.98.2:33167)
I20260926 13:42:03.971069 911 ts_manager.cc:194] Registered new tserver with Master: 3e63508cf2d6474bb7272476a8b4de85 (127.0.98.2:33167)
I20260926 13:42:03.971124 977 ts_manager.cc:194] Registered new tserver with Master: 3e63508cf2d6474bb7272476a8b4de85 (127.0.98.2:33167)
I20260926 13:42:03.971588 392 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:42:03.972482 1288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.972741 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:03.972803 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-2-root
uuid: "e85d5327b7194133aef86e19d12c9767"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:03.972852 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:03.993624 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:03.993983 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:03.994228 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:03.994697 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:03.994736 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.994767 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:03.994781 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:03.998966 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.3:42651
I20260926 13:42:03.999099 1351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.3:42651 every 8 connection(s)
I20260926 13:42:04.004662 1352 heartbeater.cc:347] Connected to a master server at 127.0.98.60:44091
I20260926 13:42:04.004732 1353 heartbeater.cc:347] Connected to a master server at 127.0.98.62:33977
I20260926 13:42:04.004753 1352 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.004825 1353 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.004988 1352 heartbeater.cc:511] Master 127.0.98.60:44091 requested a full tablet report, sending...
I20260926 13:42:04.004997 1353 heartbeater.cc:511] Master 127.0.98.62:33977 requested a full tablet report, sending...
I20260926 13:42:04.004998 1354 heartbeater.cc:347] Connected to a master server at 127.0.98.61:41179
I20260926 13:42:04.005119 1354 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.005271 1354 heartbeater.cc:511] Master 127.0.98.61:41179 requested a full tablet report, sending...
I20260926 13:42:04.005623 911 ts_manager.cc:194] Registered new tserver with Master: e85d5327b7194133aef86e19d12c9767 (127.0.98.3:42651)
I20260926 13:42:04.005749 977 ts_manager.cc:194] Registered new tserver with Master: e85d5327b7194133aef86e19d12c9767 (127.0.98.3:42651)
I20260926 13:42:04.005875 1043 ts_manager.cc:194] Registered new tserver with Master: e85d5327b7194133aef86e19d12c9767 (127.0.98.3:42651)
I20260926 13:42:04.006537 392 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007094699s
I20260926 13:42:04.053505 1084 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:42:04.053663 1084 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:04.054107 1084 leader_election.cc:288] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2e5ebc231eb4a93b1961f79de763e12 (127.0.98.62:33977), 14358fff8b70466598979e4e937c5d8e (127.0.98.61:41179)
I20260926 13:42:04.054539 987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9f98437d057463d9c176d211c44ad79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14358fff8b70466598979e4e937c5d8e" is_pre_election: true
I20260926 13:42:04.054543 921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9f98437d057463d9c176d211c44ad79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e5ebc231eb4a93b1961f79de763e12" is_pre_election: true
I20260926 13:42:04.054665 987 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9f98437d057463d9c176d211c44ad79 in term 0.
I20260926 13:42:04.054766 921 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9f98437d057463d9c176d211c44ad79 in term 0.
I20260926 13:42:04.054977 1029 leader_election.cc:302] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14358fff8b70466598979e4e937c5d8e, b9f98437d057463d9c176d211c44ad79; no voters:
I20260926 13:42:04.055123 1084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:42:04.055182 1084 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:42:04.055225 1084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:04.056097 1084 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:04.056378 1084 leader_election.cc:288] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [CANDIDATE]: Term 1 election: Requested vote from peers f2e5ebc231eb4a93b1961f79de763e12 (127.0.98.62:33977), 14358fff8b70466598979e4e937c5d8e (127.0.98.61:41179)
I20260926 13:42:04.056663 921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9f98437d057463d9c176d211c44ad79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e5ebc231eb4a93b1961f79de763e12"
I20260926 13:42:04.056669 987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9f98437d057463d9c176d211c44ad79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14358fff8b70466598979e4e937c5d8e"
I20260926 13:42:04.056762 921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:04.056795 987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 0 FOLLOWER]: Advancing to term 1
W20260926 13:42:04.057510 900 tablet.cc:2369] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 13:42:04.057648 921 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9f98437d057463d9c176d211c44ad79 in term 1.
I20260926 13:42:04.057847 987 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9f98437d057463d9c176d211c44ad79 in term 1.
I20260926 13:42:04.057902 1026 leader_election.cc:302] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9f98437d057463d9c176d211c44ad79, f2e5ebc231eb4a93b1961f79de763e12; no voters:
I20260926 13:42:04.058081 1084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:42:04.058275 1084 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 1 LEADER]: Becoming Leader. State: Replica: b9f98437d057463d9c176d211c44ad79, State: Running, Role: LEADER
I20260926 13:42:04.058413 1084 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:04.058995 1360 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9f98437d057463d9c176d211c44ad79. Latest consensus state: current_term: 1 leader_uuid: "b9f98437d057463d9c176d211c44ad79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:04.059118 1360 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:04.059371 1362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:42:04.060025 1362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:42:04.061628 921 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 1 FOLLOWER]: Refusing update from remote peer b9f98437d057463d9c176d211c44ad79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:42:04.061628 987 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Refusing update from remote peer b9f98437d057463d9c176d211c44ad79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:42:04.062050 1360 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:42:04.062206 1084 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:42:04.063371 1097 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9f98437d057463d9c176d211c44ad79. Latest consensus state: current_term: 1 leader_uuid: "b9f98437d057463d9c176d211c44ad79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:04.063500 1097 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:04.063371 1110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9f98437d057463d9c176d211c44ad79. Latest consensus state: current_term: 1 leader_uuid: "b9f98437d057463d9c176d211c44ad79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:04.063651 1110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:04.064381 1360 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9f98437d057463d9c176d211c44ad79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:04.064478 1360 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:04.064585 1360 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9f98437d057463d9c176d211c44ad79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:04.064697 1360 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:04.064919 1362 catalog_manager.cc:1347] Generated new cluster ID: bc3212fc9ea946e38e5be151835198f7
I20260926 13:42:04.064963 1362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:42:04.064973 1110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9f98437d057463d9c176d211c44ad79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:04.065065 1110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:04.065064 1097 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9f98437d057463d9c176d211c44ad79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:04.065219 1097 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:04.072072 1362 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:42:04.072903 1362 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:42:04.080849 1362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: Generated new TSK 0
I20260926 13:42:04.081004 1362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:42:04.107831 392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260926 13:42:04.110982 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:04.122908 392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260926 13:42:04.124647 392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260926 13:42:04.127935 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:04.139695 392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260926 13:42:04.141301 392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260926 13:42:04.144497 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:04.156415 392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260926 13:42:04.157995 392 master.cc:537] Master@127.0.98.62:33977 shutting down...
I20260926 13:42:04.160950 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:42:04.161103 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:04.161166 392 tablet_replica.cc:330] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: stopping tablet replica
I20260926 13:42:04.173863 392 master.cc:552] Master@127.0.98.62:33977 shutdown complete.
I20260926 13:42:04.176282 392 master.cc:537] Master@127.0.98.61:41179 shutting down...
I20260926 13:42:04.178974 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:42:04.179111 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:04.179178 392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: stopping tablet replica
I20260926 13:42:04.191623 392 master.cc:552] Master@127.0.98.61:41179 shutdown complete.
I20260926 13:42:04.194116 392 master.cc:537] Master@127.0.98.60:44091 shutting down...
I20260926 13:42:04.196712 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:42:04.196892 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:04.197026 392 tablet_replica.cc:330] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: stopping tablet replica
I20260926 13:42:04.209486 392 master.cc:552] Master@127.0.98.60:44091 shutdown complete.
I20260926 13:42:04.211846 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:04.213397 1373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.213533 1372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.213528 1375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:04.213670 392 server_base.cc:1034] running on GCE node
I20260926 13:42:04.213793 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:04.213824 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:04.213845 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430124213844 us; error 0 us; skew 500 ppm
I20260926 13:42:04.214634 392 webserver.cc:460] Webserver started at http://127.0.98.62:35919/ using document root <none> and password file <none>
I20260926 13:42:04.214746 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:04.214773 392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 13:42:04.215731 392 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:04.216563 1380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.216811 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:04.216917 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-0-root
uuid: "f2e5ebc231eb4a93b1961f79de763e12"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:04.216991 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:04.237914 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:04.238220 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:04.241978 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:33977
I20260926 13:42:04.242998 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:42:04.245122 1432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:33977 every 8 connection(s)
W20260926 13:42:04.245764 1436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.245826 1435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.245935 1438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:04.246050 392 server_base.cc:1034] running on GCE node
I20260926 13:42:04.246157 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:04.246186 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:04.246198 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430124246198 us; error 0 us; skew 500 ppm
I20260926 13:42:04.246932 392 webserver.cc:460] Webserver started at http://127.0.98.61:42097/ using document root <none> and password file <none>
I20260926 13:42:04.247037 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:04.247058 392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 13:42:04.247130 1433 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:42:04.247895 392 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:04.247969 1433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: Bootstrap starting.
I20260926 13:42:04.248590 1443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.248809 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:42:04.248858 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-1-root
uuid: "14358fff8b70466598979e4e937c5d8e"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:04.248917 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:04.253136 1433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:42:04.253484 1433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: Bootstrap complete.
I20260926 13:42:04.253887 1433 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:04.253981 1433 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e5ebc231eb4a93b1961f79de763e12, State: Initialized, Role: FOLLOWER
I20260926 13:42:04.254055 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:04.254101 1433 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:04.254347 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:04.254369 1449 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:04.254469 1449 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:04.254557 1433 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:04.256919 1471 catalog_manager.cc:1259] Loaded cluster ID: bc3212fc9ea946e38e5be151835198f7
I20260926 13:42:04.256971 1471 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: loading cluster ID for follower catalog manager: success
I20260926 13:42:04.258613 1471 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: acquiring CA information for follower catalog manager: success
I20260926 13:42:04.258927 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.61:41179
I20260926 13:42:04.258960 1508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.61:41179 every 8 connection(s)
I20260926 13:42:04.259424 1471 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:42:04.259553 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:42:04.260766 1509 sys_catalog.cc:263] Verifying existing consensus state
W20260926 13:42:04.261090 1512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.261230 1514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.261324 1511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:04.261453 392 server_base.cc:1034] running on GCE node
I20260926 13:42:04.261611 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:04.261646 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:04.261664 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430124261664 us; error 0 us; skew 500 ppm
I20260926 13:42:04.261834 1509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: Bootstrap starting.
I20260926 13:42:04.262442 392 webserver.cc:460] Webserver started at http://127.0.98.60:34973/ using document root <none> and password file <none>
I20260926 13:42:04.262566 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:04.262603 392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 13:42:04.263468 392 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:04.264264 1520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.264475 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:04.264532 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-2-root
uuid: "b9f98437d057463d9c176d211c44ad79"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:04.264591 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:04.265916 1509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:42:04.266146 1509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: Bootstrap complete.
I20260926 13:42:04.266537 1509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:04.266623 1509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14358fff8b70466598979e4e937c5d8e, State: Initialized, Role: FOLLOWER
I20260926 13:42:04.266736 1509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:04.266987 1525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:04.267092 1525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:04.267138 1509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:04.269047 1536 catalog_manager.cc:1259] Loaded cluster ID: bc3212fc9ea946e38e5be151835198f7
I20260926 13:42:04.269089 1536 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: loading cluster ID for follower catalog manager: success
I20260926 13:42:04.270100 1536 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: acquiring CA information for follower catalog manager: success
I20260926 13:42:04.270588 1536 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:42:04.275007 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:04.275280 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:04.279301 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.60:44091
I20260926 13:42:04.279472 1584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.60:44091 every 8 connection(s)
I20260926 13:42:04.279685 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 13:42:04.279724 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260926 13:42:04.279742 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260926 13:42:04.280761 1585 sys_catalog.cc:263] Verifying existing consensus state
I20260926 13:42:04.281522 1585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: Bootstrap starting.
I20260926 13:42:04.285609 1585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 13:42:04.285871 1585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: Bootstrap complete.
I20260926 13:42:04.286305 1585 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:04.286403 1585 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9f98437d057463d9c176d211c44ad79, State: Initialized, Role: FOLLOWER
I20260926 13:42:04.286514 1585 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:04.286787 1588 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:04.286885 1588 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:04.286896 1585 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:04.288707 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:42:04.288980 1599 catalog_manager.cc:1259] Loaded cluster ID: bc3212fc9ea946e38e5be151835198f7
I20260926 13:42:04.289024 1599 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: loading cluster ID for follower catalog manager: success
W20260926 13:42:04.290625 1600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.290685 1601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:04.290788 392 server_base.cc:1034] running on GCE node
W20260926 13:42:04.290917 1603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:04.290901 1599 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: acquiring CA information for follower catalog manager: success
I20260926 13:42:04.291193 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:04.291236 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:04.291250 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430124291250 us; error 0 us; skew 500 ppm
I20260926 13:42:04.291494 1599 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:42:04.292095 392 webserver.cc:460] Webserver started at http://127.0.98.1:40601/ using document root <none> and password file <none>
I20260926 13:42:04.292227 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:04.292263 392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 13:42:04.293241 392 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:04.294085 1608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.294291 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:04.294355 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-0-root
uuid: "ab856792f41e4bb09690d378cd45d833"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:04.294420 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:04.314347 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:04.314713 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:04.314977 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:04.315389 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:04.315426 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.315465 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:04.315495 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.319788 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:43369
I20260926 13:42:04.319845 1671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:43369 every 8 connection(s)
I20260926 13:42:04.320590 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:04.322512 1682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.323315 1683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.323379 1685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:04.323685 392 server_base.cc:1034] running on GCE node
I20260926 13:42:04.323853 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:04.323895 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:04.323920 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430124323919 us; error 0 us; skew 500 ppm
I20260926 13:42:04.324841 392 webserver.cc:460] Webserver started at http://127.0.98.2:38773/ using document root <none> and password file <none>
I20260926 13:42:04.324988 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:04.325035 392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 13:42:04.326297 392 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 13:42:04.326871 1674 heartbeater.cc:347] Connected to a master server at 127.0.98.61:41179
I20260926 13:42:04.326963 1674 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.327143 1674 heartbeater.cc:511] Master 127.0.98.61:41179 requested a full tablet report, sending...
I20260926 13:42:04.327121 1673 heartbeater.cc:347] Connected to a master server at 127.0.98.62:33977
I20260926 13:42:04.327133 1672 heartbeater.cc:347] Connected to a master server at 127.0.98.60:44091
I20260926 13:42:04.327227 1673 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.327244 1672 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.327270 1690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.327368 1673 heartbeater.cc:511] Master 127.0.98.62:33977 requested a full tablet report, sending...
I20260926 13:42:04.327404 1672 heartbeater.cc:511] Master 127.0.98.60:44091 requested a full tablet report, sending...
I20260926 13:42:04.327471 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:42:04.327519 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-1-root
uuid: "3e63508cf2d6474bb7272476a8b4de85"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:04.327587 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:04.327776 1473 ts_manager.cc:194] Registered new tserver with Master: ab856792f41e4bb09690d378cd45d833 (127.0.98.1:43369)
I20260926 13:42:04.327844 1397 ts_manager.cc:194] Registered new tserver with Master: ab856792f41e4bb09690d378cd45d833 (127.0.98.1:43369)
I20260926 13:42:04.329093 1549 ts_manager.cc:194] Registered new tserver with Master: ab856792f41e4bb09690d378cd45d833 (127.0.98.1:43369)
I20260926 13:42:04.344170 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:04.344499 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:04.344765 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:04.345173 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:04.345209 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.345245 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:04.345266 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.349472 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:45519
I20260926 13:42:04.349530 1753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:45519 every 8 connection(s)
I20260926 13:42:04.350272 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:04.352284 1762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.352509 1761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:04.352725 1764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:04.353446 392 server_base.cc:1034] running on GCE node
I20260926 13:42:04.353714 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:04.353762 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:04.353788 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430124353788 us; error 0 us; skew 500 ppm
I20260926 13:42:04.354836 392 webserver.cc:460] Webserver started at http://127.0.98.3:39887/ using document root <none> and password file <none>
I20260926 13:42:04.355051 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:04.355089 392 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260926 13:42:04.355950 1756 heartbeater.cc:347] Connected to a master server at 127.0.98.61:41179
I20260926 13:42:04.356053 1755 heartbeater.cc:347] Connected to a master server at 127.0.98.62:33977
I20260926 13:42:04.356067 1756 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.356145 1755 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.356408 392 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:04.356415 1756 heartbeater.cc:511] Master 127.0.98.61:41179 requested a full tablet report, sending...
I20260926 13:42:04.356438 1755 heartbeater.cc:511] Master 127.0.98.62:33977 requested a full tablet report, sending...
I20260926 13:42:04.356797 1754 heartbeater.cc:347] Connected to a master server at 127.0.98.60:44091
I20260926 13:42:04.356856 1754 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.357021 1754 heartbeater.cc:511] Master 127.0.98.60:44091 requested a full tablet report, sending...
I20260926 13:42:04.357041 1473 ts_manager.cc:194] Registered new tserver with Master: 3e63508cf2d6474bb7272476a8b4de85 (127.0.98.2:45519)
I20260926 13:42:04.357173 1397 ts_manager.cc:194] Registered new tserver with Master: 3e63508cf2d6474bb7272476a8b4de85 (127.0.98.2:45519)
I20260926 13:42:04.357239 1769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.357442 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:42:04.357509 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-2-root
uuid: "e85d5327b7194133aef86e19d12c9767"
format_stamp: "Formatted at 2026-09-26 13:42:03 on dist-test-slave-b22l"
I20260926 13:42:04.357512 1549 ts_manager.cc:194] Registered new tserver with Master: 3e63508cf2d6474bb7272476a8b4de85 (127.0.98.2:45519)
I20260926 13:42:04.357585 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskClusterRestart.1790430123331008-392-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:04.362854 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:04.363170 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:04.363389 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:04.363751 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:04.363786 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.363823 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:04.363852 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:04.368243 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.3:36955
I20260926 13:42:04.368294 1832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.3:36955 every 8 connection(s)
I20260926 13:42:04.373524 1834 heartbeater.cc:347] Connected to a master server at 127.0.98.62:33977
I20260926 13:42:04.373607 1834 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.373740 1834 heartbeater.cc:511] Master 127.0.98.62:33977 requested a full tablet report, sending...
I20260926 13:42:04.373950 1835 heartbeater.cc:347] Connected to a master server at 127.0.98.61:41179
I20260926 13:42:04.374019 1835 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.374173 1835 heartbeater.cc:511] Master 127.0.98.61:41179 requested a full tablet report, sending...
I20260926 13:42:04.374025 1833 heartbeater.cc:347] Connected to a master server at 127.0.98.60:44091
I20260926 13:42:04.374332 1397 ts_manager.cc:194] Registered new tserver with Master: e85d5327b7194133aef86e19d12c9767 (127.0.98.3:36955)
I20260926 13:42:04.374374 1833 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:04.374531 1833 heartbeater.cc:511] Master 127.0.98.60:44091 requested a full tablet report, sending...
I20260926 13:42:04.374716 1473 ts_manager.cc:194] Registered new tserver with Master: e85d5327b7194133aef86e19d12c9767 (127.0.98.3:36955)
I20260926 13:42:04.375021 1549 ts_manager.cc:194] Registered new tserver with Master: e85d5327b7194133aef86e19d12c9767 (127.0.98.3:36955)
I20260926 13:42:04.375159 392 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006460518s
I20260926 13:42:05.891942 1841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:42:05.892059 1841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:05.892571 1841 leader_election.cc:288] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f2e5ebc231eb4a93b1961f79de763e12 (127.0.98.62:33977), b9f98437d057463d9c176d211c44ad79 (127.0.98.60:44091)
I20260926 13:42:05.898253 1559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14358fff8b70466598979e4e937c5d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b9f98437d057463d9c176d211c44ad79" is_pre_election: true
I20260926 13:42:05.898298 1407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14358fff8b70466598979e4e937c5d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2e5ebc231eb4a93b1961f79de763e12" is_pre_election: true
I20260926 13:42:05.898417 1407 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14358fff8b70466598979e4e937c5d8e in term 1.
I20260926 13:42:05.898417 1559 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14358fff8b70466598979e4e937c5d8e in term 1.
I20260926 13:42:05.898720 1444 leader_election.cc:302] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14358fff8b70466598979e4e937c5d8e, b9f98437d057463d9c176d211c44ad79, f2e5ebc231eb4a93b1961f79de763e12; no voters:
I20260926 13:42:05.898933 1841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 13:42:05.898981 1841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:42:05.899012 1841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:42:05.899895 1841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:05.900189 1841 leader_election.cc:288] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [CANDIDATE]: Term 2 election: Requested vote from peers f2e5ebc231eb4a93b1961f79de763e12 (127.0.98.62:33977), b9f98437d057463d9c176d211c44ad79 (127.0.98.60:44091)
I20260926 13:42:05.900460 1407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14358fff8b70466598979e4e937c5d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f2e5ebc231eb4a93b1961f79de763e12"
I20260926 13:42:05.900509 1559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14358fff8b70466598979e4e937c5d8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b9f98437d057463d9c176d211c44ad79"
I20260926 13:42:05.900561 1407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:42:05.900635 1559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:42:05.901429 1407 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14358fff8b70466598979e4e937c5d8e in term 2.
I20260926 13:42:05.901480 1559 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14358fff8b70466598979e4e937c5d8e in term 2.
I20260926 13:42:05.901650 1444 leader_election.cc:302] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14358fff8b70466598979e4e937c5d8e, f2e5ebc231eb4a93b1961f79de763e12; no voters:
I20260926 13:42:05.901798 1841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:42:05.902016 1841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 2 LEADER]: Becoming Leader. State: Replica: 14358fff8b70466598979e4e937c5d8e, State: Running, Role: LEADER
I20260926 13:42:05.902151 1841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } }
I20260926 13:42:05.902756 1846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14358fff8b70466598979e4e937c5d8e. Latest consensus state: current_term: 2 leader_uuid: "14358fff8b70466598979e4e937c5d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e5ebc231eb4a93b1961f79de763e12" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 33977 } } peers { permanent_uuid: "14358fff8b70466598979e4e937c5d8e" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 41179 } } peers { permanent_uuid: "b9f98437d057463d9c176d211c44ad79" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 44091 } } }
I20260926 13:42:05.902880 1846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:05.903159 1848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:42:05.903877 1848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:42:05.904229 1848 catalog_manager.cc:1259] Loaded cluster ID: bc3212fc9ea946e38e5be151835198f7
I20260926 13:42:05.904270 1848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:42:05.904791 1848 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:42:05.905155 1848 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: Loaded TSK: 0
I20260926 13:42:05.905616 1848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:42:05.984418 392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260926 13:42:05.992730 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:06.005048 392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260926 13:42:06.006727 392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260926 13:42:06.010255 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:06.022014 392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260926 13:42:06.023691 392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260926 13:42:06.027205 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:06.039027 392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260926 13:42:06.040545 392 master.cc:537] Master@127.0.98.62:33977 shutting down...
I20260926 13:42:06.043666 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260926 13:42:06.043788 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f2e5ebc231eb4a93b1961f79de763e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:06.043834 392 tablet_replica.cc:330] stopping tablet replica
I20260926 13:42:06.056646 392 master.cc:552] Master@127.0.98.62:33977 shutdown complete.
I20260926 13:42:06.058772 392 master.cc:537] Master@127.0.98.61:41179 shutting down...
I20260926 13:42:06.061851 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 2 LEADER]: Raft consensus shutting down.
I20260926 13:42:06.061995 392 pending_rounds.cc:62] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: Trying to abort 1 pending ops.
I20260926 13:42:06.062068 392 pending_rounds.cc:68] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e: Aborting op as it isn't in flight: id { term: 2 index: 5 } timestamp: 7333601795696492544 op_type: NO_OP noop_request { }
I20260926 13:42:06.062162 392 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 13:42:06.062208 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 14358fff8b70466598979e4e937c5d8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:06.062284 392 tablet_replica.cc:330] stopping tablet replica
I20260926 13:42:06.064457 392 master.cc:552] Master@127.0.98.61:41179 shutdown complete.
I20260926 13:42:06.066284 392 master.cc:537] Master@127.0.98.60:44091 shutting down...
I20260926 13:42:06.069290 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260926 13:42:06.069402 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b9f98437d057463d9c176d211c44ad79 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:06.069463 392 tablet_replica.cc:330] stopping tablet replica
I20260926 13:42:06.081254 392 master.cc:552] Master@127.0.98.60:44091 shutdown complete.
[ OK ] TokenSignerITest.TskClusterRestart (2321 ms)
[ RUN ] TokenSignerITest.TskMasterLeadershipChange
I20260926 13:42:06.086723 392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:45939,127.0.98.61:42491,127.0.98.60:43309
I20260926 13:42:06.087071 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:06.088907 1852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:06.088932 1850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:06.089049 392 server_base.cc:1034] running on GCE node
W20260926 13:42:06.088958 1849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:06.089341 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:06.089381 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:06.089395 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430126089395 us; error 0 us; skew 500 ppm
I20260926 13:42:06.090267 392 webserver.cc:460] Webserver started at http://127.0.98.62:43263/ using document root <none> and password file <none>
I20260926 13:42:06.090406 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:06.090449 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:06.090517 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:06.090935 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-0-root/instance:
uuid: "8dd40fc35dc64b1d8cd6819cc30cf660"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.092298 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:06.093166 1857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.093356 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:42:06.093421 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-0-root
uuid: "8dd40fc35dc64b1d8cd6819cc30cf660"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.093495 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:06.104750 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:06.105031 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:06.108744 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:45939
I20260926 13:42:06.109915 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:42:06.111210 1909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:45939 every 8 connection(s)
I20260926 13:42:06.111877 1910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 13:42:06.112087 1912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:06.112200 1913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:06.112210 1915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:06.112766 392 server_base.cc:1034] running on GCE node
I20260926 13:42:06.112901 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:06.112939 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:06.112953 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430126112953 us; error 0 us; skew 500 ppm
I20260926 13:42:06.113363 1910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:06.113780 392 webserver.cc:460] Webserver started at http://127.0.98.61:44267/ using document root <none> and password file <none>
I20260926 13:42:06.113912 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:06.113953 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:06.114014 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:06.114467 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-1-root/instance:
uuid: "9b24d185e78e492d9112451bef9e0d84"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.116135 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 13:42:06.117081 1922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.117336 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 13:42:06.117410 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-1-root
uuid: "9b24d185e78e492d9112451bef9e0d84"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.117478 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:06.118620 1910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:06.119777 1910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.98.61:42491: Network error: Client connection negotiation failed: client connection to 127.0.98.61:42491: connect: Connection refused (error 111)
I20260926 13:42:06.125634 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:06.125916 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:06.129622 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.61:42491
I20260926 13:42:06.129642 1975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.61:42491 every 8 connection(s)
I20260926 13:42:06.130218 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:42:06.130399 1976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 13:42:06.131649 1979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:06.131739 1981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:06.131856 392 server_base.cc:1034] running on GCE node
W20260926 13:42:06.131760 1978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:06.132035 1976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:06.132153 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:06.132205 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:06.132230 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430126132230 us; error 0 us; skew 500 ppm
I20260926 13:42:06.133119 392 webserver.cc:460] Webserver started at http://127.0.98.60:41391/ using document root <none> and password file <none>
I20260926 13:42:06.133252 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:06.133302 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:06.133369 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:06.133680 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-2-root/instance:
uuid: "84caf06b743e4ddeb0ae538407261b1d"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.135207 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:42:06.136194 1987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.136395 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:42:06.136473 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-2-root
uuid: "84caf06b743e4ddeb0ae538407261b1d"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.136534 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:06.136834 1976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:06.141351 1976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:06.142447 1976 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.98.60:43309: Network error: Client connection negotiation failed: client connection to 127.0.98.60:43309: connect: Connection refused (error 111)
I20260926 13:42:06.158193 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:06.158509 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:06.162274 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.60:43309
I20260926 13:42:06.162320 2041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.60:43309 every 8 connection(s)
I20260926 13:42:06.162648 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 13:42:06.163012 2042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:42:06.164088 2042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:06.168289 2042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:06.169621 1976 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } attempt: 1
I20260926 13:42:06.171905 1910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } attempt: 1
I20260926 13:42:06.172386 2042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:06.175529 1976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: Bootstrap starting.
I20260926 13:42:06.175959 1910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:06.176549 1976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:06.177621 1976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: No bootstrap required, opened a new log
I20260926 13:42:06.178056 1976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:06.178174 1976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:06.178218 1976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b24d185e78e492d9112451bef9e0d84, State: Initialized, Role: FOLLOWER
I20260926 13:42:06.178336 1976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:06.178606 2048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:06.178718 2048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:06.178786 1976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:06.180934 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660: Bootstrap starting.
W20260926 13:42:06.181061 2059 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:06.181147 2059 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:42:06.181733 1910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:06.182714 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660: No bootstrap required, opened a new log
I20260926 13:42:06.183111 1910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:06.183197 1910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:06.183228 1910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd40fc35dc64b1d8cd6819cc30cf660, State: Initialized, Role: FOLLOWER
I20260926 13:42:06.183369 1910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:06.183651 2061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:06.183749 2061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:06.183815 1910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:06.184017 2042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d: Bootstrap starting.
I20260926 13:42:06.184818 2042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:06.185256 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260926 13:42:06.185310 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260926 13:42:06.185849 2042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d: No bootstrap required, opened a new log
W20260926 13:42:06.185866 2072 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:06.185907 2072 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:42:06.186234 2042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:06.186313 2042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:06.186333 2042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84caf06b743e4ddeb0ae538407261b1d, State: Initialized, Role: FOLLOWER
I20260926 13:42:06.186424 2042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:06.186681 2074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:06.186766 2074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:06.186781 2042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:06.188447 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:06.188498 2085 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:06.188563 2085 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 13:42:06.190158 2087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:06.190165 2086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:06.190260 2089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:06.190352 392 server_base.cc:1034] running on GCE node
I20260926 13:42:06.190538 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:06.190580 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:06.190600 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430126190600 us; error 0 us; skew 500 ppm
I20260926 13:42:06.191407 392 webserver.cc:460] Webserver started at http://127.0.98.1:34563/ using document root <none> and password file <none>
I20260926 13:42:06.191538 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:06.191579 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:06.191645 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:06.191947 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-0-root/instance:
uuid: "e8fb53c33966479bb4fb9ff8eb270070"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.193323 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:42:06.194234 2094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.194466 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:06.194525 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-0-root
uuid: "e8fb53c33966479bb4fb9ff8eb270070"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.194586 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:06.203490 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:06.203788 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:06.204037 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:06.204434 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:06.204468 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.204499 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:06.204526 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.208745 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:46231
I20260926 13:42:06.208792 2157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:46231 every 8 connection(s)
I20260926 13:42:06.209664 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:06.211824 2166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:06.211916 2165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:06.212065 392 server_base.cc:1034] running on GCE node
W20260926 13:42:06.212308 2168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:06.212774 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:06.212870 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:06.212931 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430126212931 us; error 0 us; skew 500 ppm
I20260926 13:42:06.213867 392 webserver.cc:460] Webserver started at http://127.0.98.2:40665/ using document root <none> and password file <none>
I20260926 13:42:06.214021 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:06.214079 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:06.214161 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:06.214605 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-1-root/instance:
uuid: "4ac75139df764d6185fdabf856f7f1b1"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.215056 2160 heartbeater.cc:347] Connected to a master server at 127.0.98.61:42491
I20260926 13:42:06.215138 2160 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:06.215318 2160 heartbeater.cc:511] Master 127.0.98.61:42491 requested a full tablet report, sending...
I20260926 13:42:06.215320 2158 heartbeater.cc:347] Connected to a master server at 127.0.98.60:43309
I20260926 13:42:06.215438 2158 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:06.215585 2158 heartbeater.cc:511] Master 127.0.98.60:43309 requested a full tablet report, sending...
I20260926 13:42:06.215740 2159 heartbeater.cc:347] Connected to a master server at 127.0.98.62:45939
I20260926 13:42:06.215811 2159 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:06.215957 2159 heartbeater.cc:511] Master 127.0.98.62:45939 requested a full tablet report, sending...
I20260926 13:42:06.216341 2006 ts_manager.cc:194] Registered new tserver with Master: e8fb53c33966479bb4fb9ff8eb270070 (127.0.98.1:46231)
I20260926 13:42:06.216539 1874 ts_manager.cc:194] Registered new tserver with Master: e8fb53c33966479bb4fb9ff8eb270070 (127.0.98.1:46231)
I20260926 13:42:06.216832 1940 ts_manager.cc:194] Registered new tserver with Master: e8fb53c33966479bb4fb9ff8eb270070 (127.0.98.1:46231)
I20260926 13:42:06.217301 392 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:42:06.218196 2173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.218390 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:06.218454 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-1-root
uuid: "4ac75139df764d6185fdabf856f7f1b1"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.218515 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:06.228096 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:06.228417 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:06.228690 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:06.229084 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:06.229118 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.229148 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:06.229185 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.233348 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:41327
I20260926 13:42:06.233403 2236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:41327 every 8 connection(s)
I20260926 13:42:06.234210 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:06.235922 2244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:06.236429 2245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:06.236657 2247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:06.237610 392 server_base.cc:1034] running on GCE node
I20260926 13:42:06.237792 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:06.237843 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:06.237865 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430126237865 us; error 0 us; skew 500 ppm
I20260926 13:42:06.238765 392 webserver.cc:460] Webserver started at http://127.0.98.3:42135/ using document root <none> and password file <none>
I20260926 13:42:06.238915 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:06.238963 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:06.239029 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:06.239434 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-2-root/instance:
uuid: "0a7a1a40d4b944ab85ccbab32e8d7414"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.239687 2239 heartbeater.cc:347] Connected to a master server at 127.0.98.61:42491
I20260926 13:42:06.239774 2239 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:06.239774 2238 heartbeater.cc:347] Connected to a master server at 127.0.98.62:45939
I20260926 13:42:06.239897 2238 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:06.239959 2239 heartbeater.cc:511] Master 127.0.98.61:42491 requested a full tablet report, sending...
I20260926 13:42:06.240011 2238 heartbeater.cc:511] Master 127.0.98.62:45939 requested a full tablet report, sending...
I20260926 13:42:06.240571 1874 ts_manager.cc:194] Registered new tserver with Master: 4ac75139df764d6185fdabf856f7f1b1 (127.0.98.2:41327)
I20260926 13:42:06.240568 1940 ts_manager.cc:194] Registered new tserver with Master: 4ac75139df764d6185fdabf856f7f1b1 (127.0.98.2:41327)
I20260926 13:42:06.241065 2237 heartbeater.cc:347] Connected to a master server at 127.0.98.60:43309
I20260926 13:42:06.241143 2237 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:06.241281 2237 heartbeater.cc:511] Master 127.0.98.60:43309 requested a full tablet report, sending...
I20260926 13:42:06.241281 392 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260926 13:42:06.241891 2006 ts_manager.cc:194] Registered new tserver with Master: 4ac75139df764d6185fdabf856f7f1b1 (127.0.98.2:41327)
I20260926 13:42:06.242285 2252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.242511 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:06.242575 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-2-root
uuid: "0a7a1a40d4b944ab85ccbab32e8d7414"
format_stamp: "Formatted at 2026-09-26 13:42:06 on dist-test-slave-b22l"
I20260926 13:42:06.242645 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.TskMasterLeadershipChange.1790430123331008-392-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:06.250350 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:06.250686 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:06.250932 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:06.251348 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:06.251386 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.251425 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:06.251453 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:06.255824 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.3:37857
I20260926 13:42:06.255932 2315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.3:37857 every 8 connection(s)
I20260926 13:42:06.261660 2317 heartbeater.cc:347] Connected to a master server at 127.0.98.62:45939
I20260926 13:42:06.261777 2317 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:06.261929 2316 heartbeater.cc:347] Connected to a master server at 127.0.98.60:43309
I20260926 13:42:06.261955 2317 heartbeater.cc:511] Master 127.0.98.62:45939 requested a full tablet report, sending...
I20260926 13:42:06.262004 2316 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:06.262169 2316 heartbeater.cc:511] Master 127.0.98.60:43309 requested a full tablet report, sending...
I20260926 13:42:06.262454 2318 heartbeater.cc:347] Connected to a master server at 127.0.98.61:42491
I20260926 13:42:06.262527 2318 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:06.262550 1874 ts_manager.cc:194] Registered new tserver with Master: 0a7a1a40d4b944ab85ccbab32e8d7414 (127.0.98.3:37857)
I20260926 13:42:06.262681 2318 heartbeater.cc:511] Master 127.0.98.61:42491 requested a full tablet report, sending...
I20260926 13:42:06.262876 2006 ts_manager.cc:194] Registered new tserver with Master: 0a7a1a40d4b944ab85ccbab32e8d7414 (127.0.98.3:37857)
I20260926 13:42:06.263183 1940 ts_manager.cc:194] Registered new tserver with Master: 0a7a1a40d4b944ab85ccbab32e8d7414 (127.0.98.3:37857)
I20260926 13:42:06.263289 392 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.006989287s
I20260926 13:42:06.333455 2061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:42:06.333612 2061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:06.334053 2061 leader_election.cc:288] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b24d185e78e492d9112451bef9e0d84 (127.0.98.61:42491), 84caf06b743e4ddeb0ae538407261b1d (127.0.98.60:43309)
I20260926 13:42:06.334542 1950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b24d185e78e492d9112451bef9e0d84" is_pre_election: true
I20260926 13:42:06.334542 2016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84caf06b743e4ddeb0ae538407261b1d" is_pre_election: true
I20260926 13:42:06.334724 1950 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dd40fc35dc64b1d8cd6819cc30cf660 in term 0.
I20260926 13:42:06.334724 2016 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dd40fc35dc64b1d8cd6819cc30cf660 in term 0.
I20260926 13:42:06.335053 1859 leader_election.cc:302] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dd40fc35dc64b1d8cd6819cc30cf660, 9b24d185e78e492d9112451bef9e0d84; no voters:
I20260926 13:42:06.335246 2061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:42:06.335306 2061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:42:06.335346 2061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:06.336267 2061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:06.336542 2061 leader_election.cc:288] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [CANDIDATE]: Term 1 election: Requested vote from peers 9b24d185e78e492d9112451bef9e0d84 (127.0.98.61:42491), 84caf06b743e4ddeb0ae538407261b1d (127.0.98.60:43309)
I20260926 13:42:06.336860 1950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b24d185e78e492d9112451bef9e0d84"
I20260926 13:42:06.336900 2016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84caf06b743e4ddeb0ae538407261b1d"
I20260926 13:42:06.336968 1950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:06.336968 2016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:06.337954 2016 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dd40fc35dc64b1d8cd6819cc30cf660 in term 1.
I20260926 13:42:06.337954 1950 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dd40fc35dc64b1d8cd6819cc30cf660 in term 1.
I20260926 13:42:06.338231 1859 leader_election.cc:302] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84caf06b743e4ddeb0ae538407261b1d, 8dd40fc35dc64b1d8cd6819cc30cf660; no voters:
I20260926 13:42:06.338364 2061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:42:06.338572 2061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 1 LEADER]: Becoming Leader. State: Replica: 8dd40fc35dc64b1d8cd6819cc30cf660, State: Running, Role: LEADER
I20260926 13:42:06.338716 2061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:06.339323 2324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dd40fc35dc64b1d8cd6819cc30cf660. Latest consensus state: current_term: 1 leader_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:06.339457 2324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:06.339788 2326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:42:06.340471 2326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:42:06.342099 2016 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 1 FOLLOWER]: Refusing update from remote peer 8dd40fc35dc64b1d8cd6819cc30cf660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:42:06.342142 1950 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 1 FOLLOWER]: Refusing update from remote peer 8dd40fc35dc64b1d8cd6819cc30cf660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:42:06.342422 2061 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:42:06.342628 2324 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:42:06.343839 2074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dd40fc35dc64b1d8cd6819cc30cf660. Latest consensus state: current_term: 1 leader_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:06.343860 2048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dd40fc35dc64b1d8cd6819cc30cf660. Latest consensus state: current_term: 1 leader_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:06.343962 2074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:06.343982 2048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:06.345057 2332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:06.345156 2332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:06.345280 2332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:06.345355 2332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:06.345320 2048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:06.345412 2048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:06.345527 2074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:06.345615 2074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:06.345883 2326 catalog_manager.cc:1347] Generated new cluster ID: 7f1b5ee5eaae49619c0c298fc0ac9351
I20260926 13:42:06.345934 2326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:42:06.373222 2326 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:42:06.373970 2326 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:42:06.378474 2326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660: Generated new TSK 0
I20260926 13:42:06.378638 2326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:42:06.464835 392 master.cc:537] Master@127.0.98.62:45939 shutting down...
I20260926 13:42:06.467855 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 1 LEADER]: Raft consensus shutting down.
I20260926 13:42:06.468150 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:06.468283 392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8dd40fc35dc64b1d8cd6819cc30cf660: stopping tablet replica
I20260926 13:42:06.481334 392 master.cc:552] Master@127.0.98.62:45939 shutdown complete.
I20260926 13:42:07.182024 2059 catalog_manager.cc:1259] Loaded cluster ID: 7f1b5ee5eaae49619c0c298fc0ac9351
I20260926 13:42:07.182106 2059 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: loading cluster ID for follower catalog manager: success
I20260926 13:42:07.183336 2059 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: acquiring CA information for follower catalog manager: success
I20260926 13:42:07.183902 2059 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:42:07.189095 2085 catalog_manager.cc:1259] Loaded cluster ID: 7f1b5ee5eaae49619c0c298fc0ac9351
I20260926 13:42:07.189144 2085 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d: loading cluster ID for follower catalog manager: success
I20260926 13:42:07.190167 2085 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d: acquiring CA information for follower catalog manager: success
I20260926 13:42:07.190649 2085 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 13:42:07.218204 2159 heartbeater.cc:644] Failed to heartbeat to 127.0.98.62:45939 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.98.62:45939: connect: Connection refused (error 111)
I20260926 13:42:07.879066 2339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8dd40fc35dc64b1d8cd6819cc30cf660)
I20260926 13:42:07.879204 2339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:07.879552 2340 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8dd40fc35dc64b1d8cd6819cc30cf660)
I20260926 13:42:07.879694 2339 leader_election.cc:288] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8dd40fc35dc64b1d8cd6819cc30cf660 (127.0.98.62:45939), 84caf06b743e4ddeb0ae538407261b1d (127.0.98.60:43309)
I20260926 13:42:07.879663 2340 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:07.880105 2016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b24d185e78e492d9112451bef9e0d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84caf06b743e4ddeb0ae538407261b1d" is_pre_election: true
I20260926 13:42:07.880234 2016 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b24d185e78e492d9112451bef9e0d84 in term 1.
I20260926 13:42:07.880568 1924 leader_election.cc:302] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84caf06b743e4ddeb0ae538407261b1d, 9b24d185e78e492d9112451bef9e0d84; no voters:
I20260926 13:42:07.880745 2339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 13:42:07.880811 2339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8dd40fc35dc64b1d8cd6819cc30cf660)
I20260926 13:42:07.880858 2339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:42:07.880942 2340 leader_election.cc:288] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8dd40fc35dc64b1d8cd6819cc30cf660 (127.0.98.62:45939), 9b24d185e78e492d9112451bef9e0d84 (127.0.98.61:42491)
I20260926 13:42:07.881228 1950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "84caf06b743e4ddeb0ae538407261b1d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9b24d185e78e492d9112451bef9e0d84" is_pre_election: true
W20260926 13:42:07.881547 1990 leader_election.cc:334] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8dd40fc35dc64b1d8cd6819cc30cf660 (127.0.98.62:45939): Network error: Client connection negotiation failed: client connection to 127.0.98.62:45939: connect: Connection refused (error 111)
W20260926 13:42:07.882200 1925 leader_election.cc:334] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8dd40fc35dc64b1d8cd6819cc30cf660 (127.0.98.62:45939): Network error: Client connection negotiation failed: client connection to 127.0.98.62:45939: connect: Connection refused (error 111)
I20260926 13:42:07.948362 2339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:07.948767 1950 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84caf06b743e4ddeb0ae538407261b1d in current term 2: Already voted for candidate 9b24d185e78e492d9112451bef9e0d84 in this term.
I20260926 13:42:07.949147 2339 leader_election.cc:288] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [CANDIDATE]: Term 2 election: Requested vote from peers 8dd40fc35dc64b1d8cd6819cc30cf660 (127.0.98.62:45939), 84caf06b743e4ddeb0ae538407261b1d (127.0.98.60:43309)
I20260926 13:42:07.949501 2016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b24d185e78e492d9112451bef9e0d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84caf06b743e4ddeb0ae538407261b1d"
I20260926 13:42:07.949609 2016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 1 FOLLOWER]: Advancing to term 2
I20260926 13:42:07.949831 1989 leader_election.cc:302] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84caf06b743e4ddeb0ae538407261b1d; no voters: 8dd40fc35dc64b1d8cd6819cc30cf660, 9b24d185e78e492d9112451bef9e0d84
W20260926 13:42:07.950387 1925 leader_election.cc:334] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8dd40fc35dc64b1d8cd6819cc30cf660 (127.0.98.62:45939): Network error: Client connection negotiation failed: client connection to 127.0.98.62:45939: connect: Connection refused (error 111)
I20260926 13:42:07.950703 2016 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b24d185e78e492d9112451bef9e0d84 in term 2.
I20260926 13:42:07.950817 2340 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 13:42:07.950982 1924 leader_election.cc:302] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84caf06b743e4ddeb0ae538407261b1d, 9b24d185e78e492d9112451bef9e0d84; no voters: 8dd40fc35dc64b1d8cd6819cc30cf660
I20260926 13:42:07.951128 2339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 13:42:07.951339 2339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 2 LEADER]: Becoming Leader. State: Replica: 9b24d185e78e492d9112451bef9e0d84, State: Running, Role: LEADER
I20260926 13:42:07.951501 2339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } }
I20260926 13:42:07.952118 2345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b24d185e78e492d9112451bef9e0d84. Latest consensus state: current_term: 2 leader_uuid: "9b24d185e78e492d9112451bef9e0d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dd40fc35dc64b1d8cd6819cc30cf660" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 45939 } } peers { permanent_uuid: "9b24d185e78e492d9112451bef9e0d84" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 42491 } } peers { permanent_uuid: "84caf06b743e4ddeb0ae538407261b1d" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 43309 } } }
I20260926 13:42:07.952239 2345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:07.952482 2347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:42:07.953275 2347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:42:07.953631 2347 catalog_manager.cc:1259] Loaded cluster ID: 7f1b5ee5eaae49619c0c298fc0ac9351
I20260926 13:42:07.953676 2347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:42:07.954185 2347 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:42:07.954542 2347 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: Loaded TSK: 0
I20260926 13:42:07.954767 2347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:42:08.050364 392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260926 13:42:08.053608 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:08.065490 392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260926 13:42:08.067137 392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260926 13:42:08.070238 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:08.082132 392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260926 13:42:08.083722 392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260926 13:42:08.086707 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:42:08.098876 392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260926 13:42:08.100483 392 master.cc:537] Master@127.0.98.61:42491 shutting down...
I20260926 13:42:08.103291 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 2 LEADER]: Raft consensus shutting down.
I20260926 13:42:08.103466 392 pending_rounds.cc:62] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: Trying to abort 1 pending ops.
I20260926 13:42:08.103493 392 pending_rounds.cc:68] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: Aborting op as it isn't in flight: id { term: 2 index: 5 } timestamp: 7333601804090732544 op_type: NO_OP noop_request { }
I20260926 13:42:08.103556 392 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 13:42:08.103597 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:08.103672 392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9b24d185e78e492d9112451bef9e0d84: stopping tablet replica
I20260926 13:42:08.115839 392 master.cc:552] Master@127.0.98.61:42491 shutdown complete.
I20260926 13:42:08.118012 392 master.cc:537] Master@127.0.98.60:43309 shutting down...
I20260926 13:42:08.120926 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260926 13:42:08.121059 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260926 13:42:08.121125 392 tablet_replica.cc:330] T 00000000000000000000000000000000 P 84caf06b743e4ddeb0ae538407261b1d: stopping tablet replica
I20260926 13:42:08.133143 392 master.cc:552] Master@127.0.98.60:43309 shutdown complete.
[ OK ] TokenSignerITest.TskMasterLeadershipChange (2051 ms)
[ RUN ] TokenSignerITest.AuthnTokenLifecycle
I20260926 13:42:08.138598 392 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.98.62:39415,127.0.98.61:33993,127.0.98.60:39193
I20260926 13:42:08.138965 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:08.140528 2349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:08.140522 2348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:08.140524 2351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:08.140807 392 server_base.cc:1034] running on GCE node
I20260926 13:42:08.140940 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:08.140972 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:08.140992 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430128140992 us; error 0 us; skew 500 ppm
I20260926 13:42:08.141772 392 webserver.cc:460] Webserver started at http://127.0.98.62:46415/ using document root <none> and password file <none>
I20260926 13:42:08.141920 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:08.141963 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:08.142030 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:08.142403 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-0-root/instance:
uuid: "f992bec6108d45b898faafc14cbbc18d"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.143738 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:08.144707 2356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.144927 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:08.144989 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-0-root
uuid: "f992bec6108d45b898faafc14cbbc18d"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.145054 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:08.158743 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:08.159214 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:08.163131 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.62:39415
I20260926 13:42:08.164237 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:42:08.166184 2408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.62:39415 every 8 connection(s)
I20260926 13:42:08.167016 2409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 13:42:08.167140 2412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:08.167315 392 server_base.cc:1034] running on GCE node
W20260926 13:42:08.167412 2414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:08.167557 2411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:08.167719 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:08.167769 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:08.167788 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430128167787 us; error 0 us; skew 500 ppm
I20260926 13:42:08.168488 2409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:08.168630 392 webserver.cc:460] Webserver started at http://127.0.98.61:35457/ using document root <none> and password file <none>
I20260926 13:42:08.168753 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:08.168802 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:08.168862 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:08.169216 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-1-root/instance:
uuid: "868f88ad995a4b6a8cbfd935c537597a"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.170620 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 13:42:08.171502 2421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.171729 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:08.171798 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-1-root
uuid: "868f88ad995a4b6a8cbfd935c537597a"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.171864 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:08.173420 2409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:08.174459 2409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.98.61:33993: Network error: Client connection negotiation failed: client connection to 127.0.98.61:33993: connect: Connection refused (error 111)
I20260926 13:42:08.182526 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:08.182842 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:08.186509 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.61:33993
I20260926 13:42:08.186537 2474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.61:33993 every 8 connection(s)
I20260926 13:42:08.187098 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 13:42:08.187237 2475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:42:08.188536 2475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:08.188800 2478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:08.189095 2477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:08.189194 392 server_base.cc:1034] running on GCE node
W20260926 13:42:08.189399 2480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:08.189766 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:08.189811 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:08.189831 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430128189830 us; error 0 us; skew 500 ppm
I20260926 13:42:08.190670 392 webserver.cc:460] Webserver started at http://127.0.98.60:43215/ using document root <none> and password file <none>
I20260926 13:42:08.190811 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:08.190869 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:08.190927 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:08.191308 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-2-root/instance:
uuid: "6a920040626144ad8a48653de6c7bdf6"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.193053 392 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 13:42:08.193428 2475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:08.194027 2486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.194268 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 13:42:08.194334 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-2-root
uuid: "6a920040626144ad8a48653de6c7bdf6"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.194393 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:08.198287 2475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } has no permanent_uuid. Determining permanent_uuid...
W20260926 13:42:08.199375 2475 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.98.60:39193: Network error: Client connection negotiation failed: client connection to 127.0.98.60:39193: connect: Connection refused (error 111)
I20260926 13:42:08.214775 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:08.215162 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:08.219456 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.60:39193
I20260926 13:42:08.219524 2540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.60:39193 every 8 connection(s)
I20260926 13:42:08.219878 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260926 13:42:08.220235 2541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 13:42:08.221386 2541 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:08.225382 2541 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:08.225584 2409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } attempt: 1
I20260926 13:42:08.229707 2409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:08.229902 2541 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } has no permanent_uuid. Determining permanent_uuid...
I20260926 13:42:08.235453 2409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: Bootstrap starting.
I20260926 13:42:08.235453 2541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6: Bootstrap starting.
I20260926 13:42:08.236507 2409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:08.236507 2541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:08.237679 2409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: No bootstrap required, opened a new log
I20260926 13:42:08.237814 2541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6: No bootstrap required, opened a new log
I20260926 13:42:08.238183 2409 raft_consensus.cc:348] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } }
I20260926 13:42:08.238183 2541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } }
I20260926 13:42:08.238327 2541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:08.238318 2409 raft_consensus.cc:374] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:08.238406 2409 raft_consensus.cc:729] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f992bec6108d45b898faafc14cbbc18d, State: Initialized, Role: FOLLOWER
I20260926 13:42:08.238397 2541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a920040626144ad8a48653de6c7bdf6, State: Initialized, Role: FOLLOWER
I20260926 13:42:08.238559 2541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } }
I20260926 13:42:08.238567 2409 consensus_queue.cc:260] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } }
I20260926 13:42:08.238899 2549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } } }
I20260926 13:42:08.239017 2549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:08.239061 2409 sys_catalog.cc:564] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:08.239177 2550 sys_catalog.cc:455] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } } }
I20260926 13:42:08.239260 2550 sys_catalog.cc:458] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:08.239061 2541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:08.240554 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20260926 13:42:08.241153 2561 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:08.241217 2561 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 13:42:08.242443 2572 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:08.242504 2572 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 13:42:08.244521 2475 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } attempt: 1
I20260926 13:42:08.252496 2475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: Bootstrap starting.
I20260926 13:42:08.253324 2475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: Neither blocks nor log segments found. Creating new log.
I20260926 13:42:08.254348 2475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: No bootstrap required, opened a new log
I20260926 13:42:08.254768 2475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } }
I20260926 13:42:08.254858 2475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 13:42:08.254890 2475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868f88ad995a4b6a8cbfd935c537597a, State: Initialized, Role: FOLLOWER
I20260926 13:42:08.255034 2475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } }
I20260926 13:42:08.255330 2574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } } }
I20260926 13:42:08.255420 2574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:08.255451 2475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 13:42:08.256873 392 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260926 13:42:08.257357 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:08.257440 2585 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 13:42:08.257491 2585 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 13:42:08.259244 2587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:08.259244 2586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:08.259385 392 server_base.cc:1034] running on GCE node
W20260926 13:42:08.259394 2589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:08.259656 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:08.259702 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:08.259724 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430128259723 us; error 0 us; skew 500 ppm
I20260926 13:42:08.260700 392 webserver.cc:460] Webserver started at http://127.0.98.1:41609/ using document root <none> and password file <none>
I20260926 13:42:08.260869 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:08.260926 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:08.260998 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:08.261348 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-0-root/instance:
uuid: "8832ce8d90c14877b770cf6a239b2964"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.262667 392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260926 13:42:08.263496 2594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.263679 392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260926 13:42:08.263741 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-0-root
uuid: "8832ce8d90c14877b770cf6a239b2964"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.263800 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:08.268896 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:08.269199 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:08.269439 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:08.269822 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:08.269865 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.269903 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:08.269932 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.274406 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.1:35037
I20260926 13:42:08.274464 2657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.1:35037 every 8 connection(s)
I20260926 13:42:08.275310 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:08.277419 2666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:08.277725 392 server_base.cc:1034] running on GCE node
W20260926 13:42:08.278157 2668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:08.279161 2664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:08.279417 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:08.279464 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:08.279484 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430128279483 us; error 0 us; skew 500 ppm
I20260926 13:42:08.280277 392 webserver.cc:460] Webserver started at http://127.0.98.2:42175/ using document root <none> and password file <none>
I20260926 13:42:08.280396 2658 heartbeater.cc:347] Connected to a master server at 127.0.98.60:39193
I20260926 13:42:08.280427 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:08.280503 2658 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:08.280517 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:08.280583 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:08.280709 2658 heartbeater.cc:511] Master 127.0.98.60:39193 requested a full tablet report, sending...
I20260926 13:42:08.281021 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-1-root/instance:
uuid: "89947a39699c46fcaf8d6f8affe468bd"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.281345 2505 ts_manager.cc:194] Registered new tserver with Master: 8832ce8d90c14877b770cf6a239b2964 (127.0.98.1:35037)
I20260926 13:42:08.281755 2659 heartbeater.cc:347] Connected to a master server at 127.0.98.62:39415
I20260926 13:42:08.281826 2659 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:08.281975 2659 heartbeater.cc:511] Master 127.0.98.62:39415 requested a full tablet report, sending...
I20260926 13:42:08.282529 2660 heartbeater.cc:347] Connected to a master server at 127.0.98.61:33993
I20260926 13:42:08.282603 2660 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:08.282660 2373 ts_manager.cc:194] Registered new tserver with Master: 8832ce8d90c14877b770cf6a239b2964 (127.0.98.1:35037)
I20260926 13:42:08.282755 2660 heartbeater.cc:511] Master 127.0.98.61:33993 requested a full tablet report, sending...
I20260926 13:42:08.283131 392 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260926 13:42:08.283322 2439 ts_manager.cc:194] Registered new tserver with Master: 8832ce8d90c14877b770cf6a239b2964 (127.0.98.1:35037)
I20260926 13:42:08.284123 2673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.284336 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:42:08.284399 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-1-root
uuid: "89947a39699c46fcaf8d6f8affe468bd"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.284459 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:08.310582 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:08.310992 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:08.311292 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:08.311760 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:08.311798 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.311841 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:08.311882 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.316412 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.2:36721
I20260926 13:42:08.316534 2736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.2:36721 every 8 connection(s)
I20260926 13:42:08.317343 392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 13:42:08.320093 2745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:08.320181 2747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 13:42:08.320365 2744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 13:42:08.320772 392 server_base.cc:1034] running on GCE node
I20260926 13:42:08.320961 392 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260926 13:42:08.321000 392 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260926 13:42:08.321019 392 hybrid_clock.cc:648] HybridClock initialized: now 1790430128321019 us; error 0 us; skew 500 ppm
I20260926 13:42:08.322252 392 webserver.cc:460] Webserver started at http://127.0.98.3:33195/ using document root <none> and password file <none>
I20260926 13:42:08.322412 392 fs_manager.cc:359] Metadata directory not provided
I20260926 13:42:08.322469 392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 13:42:08.322559 392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 13:42:08.322739 2737 heartbeater.cc:347] Connected to a master server at 127.0.98.60:39193
I20260926 13:42:08.322829 2737 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:08.323006 2737 heartbeater.cc:511] Master 127.0.98.60:39193 requested a full tablet report, sending...
I20260926 13:42:08.323093 392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-2-root/instance:
uuid: "99124cc11d16404687a2e1e96cbb92fb"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.323103 2738 heartbeater.cc:347] Connected to a master server at 127.0.98.62:39415
I20260926 13:42:08.323240 2738 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:08.323395 2738 heartbeater.cc:511] Master 127.0.98.62:39415 requested a full tablet report, sending...
I20260926 13:42:08.323626 2505 ts_manager.cc:194] Registered new tserver with Master: 89947a39699c46fcaf8d6f8affe468bd (127.0.98.2:36721)
I20260926 13:42:08.323805 2739 heartbeater.cc:347] Connected to a master server at 127.0.98.61:33993
I20260926 13:42:08.323874 2739 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:08.324034 2739 heartbeater.cc:511] Master 127.0.98.61:33993 requested a full tablet report, sending...
I20260926 13:42:08.324200 2373 ts_manager.cc:194] Registered new tserver with Master: 89947a39699c46fcaf8d6f8affe468bd (127.0.98.2:36721)
I20260926 13:42:08.324591 2439 ts_manager.cc:194] Registered new tserver with Master: 89947a39699c46fcaf8d6f8affe468bd (127.0.98.2:36721)
I20260926 13:42:08.324903 392 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260926 13:42:08.325830 2752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.326038 392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260926 13:42:08.326102 392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-2-root
uuid: "99124cc11d16404687a2e1e96cbb92fb"
format_stamp: "Formatted at 2026-09-26 13:42:08 on dist-test-slave-b22l"
I20260926 13:42:08.326161 392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskYp2dUY/test-tmp/token_signer-itest.0.TokenSignerITest.AuthnTokenLifecycle.1790430123331008-392-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 13:42:08.347944 392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 13:42:08.348382 392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 13:42:08.348706 392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 13:42:08.349193 392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 13:42:08.349232 392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.349279 392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 13:42:08.349308 392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 13:42:08.354363 392 rpc_server.cc:307] RPC server started. Bound to: 127.0.98.3:45815
I20260926 13:42:08.354400 2815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.98.3:45815 every 8 connection(s)
I20260926 13:42:08.359901 2818 heartbeater.cc:347] Connected to a master server at 127.0.98.61:33993
I20260926 13:42:08.359967 2817 heartbeater.cc:347] Connected to a master server at 127.0.98.62:39415
I20260926 13:42:08.360061 2817 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:08.360064 2818 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:08.360248 2817 heartbeater.cc:511] Master 127.0.98.62:39415 requested a full tablet report, sending...
I20260926 13:42:08.360255 2818 heartbeater.cc:511] Master 127.0.98.61:33993 requested a full tablet report, sending...
I20260926 13:42:08.360896 2816 heartbeater.cc:347] Connected to a master server at 127.0.98.60:39193
I20260926 13:42:08.360939 2373 ts_manager.cc:194] Registered new tserver with Master: 99124cc11d16404687a2e1e96cbb92fb (127.0.98.3:45815)
I20260926 13:42:08.360947 2439 ts_manager.cc:194] Registered new tserver with Master: 99124cc11d16404687a2e1e96cbb92fb (127.0.98.3:45815)
I20260926 13:42:08.361212 2816 heartbeater.cc:464] Registering TS with master...
I20260926 13:42:08.361368 2816 heartbeater.cc:511] Master 127.0.98.60:39193 requested a full tablet report, sending...
I20260926 13:42:08.361902 2505 ts_manager.cc:194] Registered new tserver with Master: 99124cc11d16404687a2e1e96cbb92fb (127.0.98.3:45815)
I20260926 13:42:08.362848 392 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007991303s
I20260926 13:42:08.395147 2549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 13:42:08.395280 2549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } }
I20260926 13:42:08.395699 2549 leader_election.cc:288] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f992bec6108d45b898faafc14cbbc18d (127.0.98.62:39415), 868f88ad995a4b6a8cbfd935c537597a (127.0.98.61:33993)
I20260926 13:42:08.396149 2383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a920040626144ad8a48653de6c7bdf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f992bec6108d45b898faafc14cbbc18d" is_pre_election: true
I20260926 13:42:08.396149 2449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a920040626144ad8a48653de6c7bdf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868f88ad995a4b6a8cbfd935c537597a" is_pre_election: true
I20260926 13:42:08.396339 2449 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a920040626144ad8a48653de6c7bdf6 in term 0.
I20260926 13:42:08.396339 2383 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a920040626144ad8a48653de6c7bdf6 in term 0.
I20260926 13:42:08.396713 2491 leader_election.cc:302] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a920040626144ad8a48653de6c7bdf6, f992bec6108d45b898faafc14cbbc18d; no voters:
I20260926 13:42:08.396883 2549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 13:42:08.396950 2549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 13:42:08.396979 2549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:08.451267 2549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } }
I20260926 13:42:08.451864 2549 leader_election.cc:288] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [CANDIDATE]: Term 1 election: Requested vote from peers f992bec6108d45b898faafc14cbbc18d (127.0.98.62:39415), 868f88ad995a4b6a8cbfd935c537597a (127.0.98.61:33993)
I20260926 13:42:08.452255 2449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a920040626144ad8a48653de6c7bdf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868f88ad995a4b6a8cbfd935c537597a"
I20260926 13:42:08.452255 2383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a920040626144ad8a48653de6c7bdf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f992bec6108d45b898faafc14cbbc18d"
I20260926 13:42:08.452399 2449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:08.452440 2383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [term 0 FOLLOWER]: Advancing to term 1
I20260926 13:42:08.453447 2383 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a920040626144ad8a48653de6c7bdf6 in term 1.
I20260926 13:42:08.453570 2449 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a920040626144ad8a48653de6c7bdf6 in term 1.
I20260926 13:42:08.453747 2491 leader_election.cc:302] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a920040626144ad8a48653de6c7bdf6, f992bec6108d45b898faafc14cbbc18d; no voters:
I20260926 13:42:08.453918 2549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 13:42:08.454136 2549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [term 1 LEADER]: Becoming Leader. State: Replica: 6a920040626144ad8a48653de6c7bdf6, State: Running, Role: LEADER
I20260926 13:42:08.454288 2549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } }
I20260926 13:42:08.454998 2824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a920040626144ad8a48653de6c7bdf6. Latest consensus state: current_term: 1 leader_uuid: "6a920040626144ad8a48653de6c7bdf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } } }
I20260926 13:42:08.455155 2824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:08.455514 2826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 13:42:08.456187 2826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 13:42:08.457991 2383 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [term 1 FOLLOWER]: Refusing update from remote peer 6a920040626144ad8a48653de6c7bdf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:42:08.457991 2449 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [term 1 FOLLOWER]: Refusing update from remote peer 6a920040626144ad8a48653de6c7bdf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 13:42:08.458367 2824 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:42:08.458508 2549 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 13:42:08.459748 2550 sys_catalog.cc:455] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a920040626144ad8a48653de6c7bdf6. Latest consensus state: current_term: 1 leader_uuid: "6a920040626144ad8a48653de6c7bdf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } } }
I20260926 13:42:08.459872 2550 sys_catalog.cc:458] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:08.459765 2574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a920040626144ad8a48653de6c7bdf6. Latest consensus state: current_term: 1 leader_uuid: "6a920040626144ad8a48653de6c7bdf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } } }
I20260926 13:42:08.460039 2574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:08.460996 2549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a920040626144ad8a48653de6c7bdf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } } }
I20260926 13:42:08.461095 2549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:08.461194 2549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a920040626144ad8a48653de6c7bdf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } } }
I20260926 13:42:08.461267 2549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6 [sys.catalog]: This master's current role is: LEADER
I20260926 13:42:08.461230 2574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a920040626144ad8a48653de6c7bdf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } } }
I20260926 13:42:08.461331 2574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:08.461515 2550 sys_catalog.cc:455] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a920040626144ad8a48653de6c7bdf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f992bec6108d45b898faafc14cbbc18d" member_type: VOTER last_known_addr { host: "127.0.98.62" port: 39415 } } peers { permanent_uuid: "868f88ad995a4b6a8cbfd935c537597a" member_type: VOTER last_known_addr { host: "127.0.98.61" port: 33993 } } peers { permanent_uuid: "6a920040626144ad8a48653de6c7bdf6" member_type: VOTER last_known_addr { host: "127.0.98.60" port: 39193 } } }
I20260926 13:42:08.461606 2550 sys_catalog.cc:458] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 13:42:08.461621 2826 catalog_manager.cc:1347] Generated new cluster ID: 29cac327d35342b3868019e1920b3332
I20260926 13:42:08.461665 2826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 13:42:08.471735 2826 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 13:42:08.472311 2826 catalog_manager.cc:1504] Loading token signing keys...
I20260926 13:42:08.479132 2826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6: Generated new TSK 0
I20260926 13:42:08.479308 2826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 13:42:09.242141 2561 catalog_manager.cc:1259] Loaded cluster ID: 29cac327d35342b3868019e1920b3332
I20260926 13:42:09.242221 2561 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: loading cluster ID for follower catalog manager: success
I20260926 13:42:09.243402 2561 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: acquiring CA information for follower catalog manager: success
I20260926 13:42:09.243920 2561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:42:09.258219 2585 catalog_manager.cc:1259] Loaded cluster ID: 29cac327d35342b3868019e1920b3332
I20260926 13:42:09.258289 2585 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: loading cluster ID for follower catalog manager: success
I20260926 13:42:09.259404 2585 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: acquiring CA information for follower catalog manager: success
I20260926 13:42:09.259960 2585 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:42:09.283648 2505 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41374
I20260926 13:42:09.325806 2505 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41388
I20260926 13:42:09.363266 2505 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41398
I20260926 13:42:10.284694 2658 heartbeater.cc:503] Master 127.0.98.60:39193 was elected leader, sending a full tablet report...
I20260926 13:42:10.326628 2737 heartbeater.cc:503] Master 127.0.98.60:39193 was elected leader, sending a full tablet report...
I20260926 13:42:10.364142 2816 heartbeater.cc:503] Master 127.0.98.60:39193 was elected leader, sending a full tablet report...
I20260926 13:42:19.247328 2561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:42:19.263517 2585 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 13:42:28.255928 2572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6: Generated new TSK 1
I20260926 13:42:29.250949 2561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 1
I20260926 13:42:29.267122 2585 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 1
I20260926 13:42:39.254935 2561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 1
I20260926 13:42:39.270828 2585 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 1
I20260926 13:42:49.258841 2561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 1
I20260926 13:42:49.272547 2572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a920040626144ad8a48653de6c7bdf6: Generated new TSK 2
I20260926 13:42:49.274227 2585 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 2
I20260926 13:42:59.262871 2561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 2
I20260926 13:42:59.277657 2585 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 868f88ad995a4b6a8cbfd935c537597a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 2
I20260926 13:43:08.114804 392 tablet_server.cc:178] TabletServer@127.0.98.1:0 shutting down...
I20260926 13:43:08.118818 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:43:08.132025 392 tablet_server.cc:194] TabletServer@127.0.98.1:0 shutdown complete.
I20260926 13:43:08.133865 392 tablet_server.cc:178] TabletServer@127.0.98.2:0 shutting down...
I20260926 13:43:08.137504 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:43:08.149308 392 tablet_server.cc:194] TabletServer@127.0.98.2:0 shutdown complete.
I20260926 13:43:08.150872 392 tablet_server.cc:178] TabletServer@127.0.98.3:0 shutting down...
I20260926 13:43:08.154541 392 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260926 13:43:08.166989 392 tablet_server.cc:194] TabletServer@127.0.98.3:0 shutdown complete.
I20260926 13:43:08.168828 392 master.cc:537] Master@127.0.98.62:39415 shutting down...
I20260926 13:43:08.172953 392 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 13:43:08.173105 392 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 13:43:08.173169 392 tablet_replica.cc:330] T 00000000000000000000000000000000 P f992bec6108d45b898faafc14cbbc18d: stopping tablet replica
F20260926 13:43:08.182272 2404 diagnostic_socket.cc:265] Check failed: fd_ >= 0 (-1 vs. 0)
*** Check failure stack trace: ***
*** Aborted at 1790430188 (unix time) try "date -d @1790430188" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800000188) received by PID 392 (TID 0x7f270ca11700) from PID 392; stack trace: ***
@ 0x7f2729ca8a46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
@ 0x7f272c426980 (unknown) at ??:0
@ 0x7f272775de87 gsignal at ??:0
@ 0x7f272775f7f1 abort at ??:0
@ 0x7f2729c9a9ad google::LogMessage::Fail() at ??:0
@ 0x7f2729c9d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f2729c9a4de google::LogMessage::Flush() at ??:0
@ 0x7f2729c9ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f272b2973eb kudu::DiagnosticSocket::ReceiveResponse() at ??:0
@ 0x7f272b2968da kudu::DiagnosticSocket::Query() at ??:0
@ 0x7f272f4d7018 kudu::rpc::AcceptorPool::GetPendingConnectionsNum() at ??:0
@ 0x7f272f50c55a kudu::rpc::Messenger::GetPendingConnectionsNum() at ??:0
@ 0x7f272f50cd57 _ZZN4kudu3rpc9Messenger15AddAcceptorPoolERKNS_8SockaddrEPSt10shared_ptrINS0_12AcceptorPoolEEENKUlvE_clEv at ??:0
@ 0x7f272f50f918 _ZNSt17_Function_handlerIFivEZN4kudu3rpc9Messenger15AddAcceptorPoolERKNS1_8SockaddrEPSt10shared_ptrINS2_12AcceptorPoolEEEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f2733b334c2 std::function<>::operator()() at ??:0
@ 0x7f2733b332e8 kudu::FunctionGauge<>::value() at ??:0
@ 0x7f2733b330b4 kudu::FunctionGauge<>::WriteValue() at ??:0
@ 0x7f272b27d86a kudu::Gauge::WriteAsJson() at ??:0
@ 0x7f272b284149 kudu::WriteMetricsToJson<>() at ??:0
@ 0x7f272b27a435 kudu::MetricEntity::WriteAsJson() at ??:0
@ 0x7f272b27bb47 kudu::MetricRegistry::WriteAsJson() at ??:0
@ 0x7f2730b21d25 kudu::server::DiagnosticsLog::LogMetrics() at ??:0
@ 0x7f2730b20f22 kudu::server::DiagnosticsLog::RunThread() at ??:0
@ 0x7f2730b20557 _ZZN4kudu6server14DiagnosticsLog5StartEvENKUlvE_clEv at ??:0
@ 0x7f2730b221ed _ZNSt17_Function_handlerIFvvEZN4kudu6server14DiagnosticsLog5StartEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f2734139170 std::function<>::operator()() at ??:0
@ 0x7f272b2ed512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f272c41b6db start_thread at ??:0
@ 0x7f272784061f clone at ??:0