[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:42:51 dist-test-slave-w8nh krb5kdc[6440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:42:51 dist-test-slave-w8nh krb5kdc[6440](info): set up 2 sockets
Sep 25 00:42:51 dist-test-slave-w8nh krb5kdc[6440](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 00:42:52.456887 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:42:52 dist-test-slave-w8nh krb5kdc[6440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:42:52.589779 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:33563
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:33563
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:52.705205 6453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:52.705578 6453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:52.705652 6453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:52.710016 6453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:42:52.710124 6453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:52.710160 6453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:52.710210 6453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:42:52.710250 6453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:42:52.715886 6453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:33563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:33563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:52.717116 6453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:52.718245 6453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:52.727650 6458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:52.727708 6453 server_base.cc:1034] running on GCE node
W20260925 00:42:52.727736 6459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:52.727677 6461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:52.728577 6453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:52.729849 6453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:52.731025 6453 hybrid_clock.cc:648] HybridClock initialized: now 1790296972731009 us; error 36 us; skew 500 ppm
Sep 25 00:42:52 dist-test-slave-w8nh krb5kdc[6440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296972, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:52.734118 6453 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:42:52.735200 6453 webserver.cc:460] Webserver started at http://127.6.72.126:40909/ using document root <none> and password file <none>
I20260925 00:42:52.735831 6453 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:52.735932 6453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:52.736186 6453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:52.738430 6453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "a04876fd00b144ce8a204ea9f0b7fe63"
format_stamp: "Formatted at 2026-09-25 00:42:52 on dist-test-slave-w8nh"
server_key: "1947e3b7e5191bf5ab95184cbdff692d"
server_key_iv: "96b01a8e3c98b9f48b921a9d2c800719"
server_key_version: "encryptionkey@0"
I20260925 00:42:52.739061 6453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "a04876fd00b144ce8a204ea9f0b7fe63"
format_stamp: "Formatted at 2026-09-25 00:42:52 on dist-test-slave-w8nh"
server_key: "1947e3b7e5191bf5ab95184cbdff692d"
server_key_iv: "96b01a8e3c98b9f48b921a9d2c800719"
server_key_version: "encryptionkey@0"
I20260925 00:42:52.743197 6453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 00:42:52.745893 6468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:52.747304 6453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 00:42:52.747452 6453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "a04876fd00b144ce8a204ea9f0b7fe63"
format_stamp: "Formatted at 2026-09-25 00:42:52 on dist-test-slave-w8nh"
server_key: "1947e3b7e5191bf5ab95184cbdff692d"
server_key_iv: "96b01a8e3c98b9f48b921a9d2c800719"
server_key_version: "encryptionkey@0"
I20260925 00:42:52.747575 6453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:52.759296 6453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:52.762439 6453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:52.762665 6453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:52.770133 6453 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:33563
I20260925 00:42:52.770160 6520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:33563 every 8 connection(s)
I20260925 00:42:52.771131 6453 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:42:52.773954 6521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:52.777362 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 6453
I20260925 00:42:52.777537 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:42:52.779173 6433 external_mini_cluster.cc:1428] Setting key 336dc99dcf3331df81bf326697d54307
I20260925 00:42:52.780722 6521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63: Bootstrap starting.
I20260925 00:42:52.783596 6521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:52.784572 6521 log.cc:824] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63: Log is configured to *not* fsync() on all Append() calls
I20260925 00:42:52.787034 6521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63: No bootstrap required, opened a new log
I20260925 00:42:52.791949 6521 raft_consensus.cc:348] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04876fd00b144ce8a204ea9f0b7fe63" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33563 } }
I20260925 00:42:52.792294 6521 raft_consensus.cc:374] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:52.792362 6521 raft_consensus.cc:729] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a04876fd00b144ce8a204ea9f0b7fe63, State: Initialized, Role: FOLLOWER
I20260925 00:42:52.792897 6521 consensus_queue.cc:260] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04876fd00b144ce8a204ea9f0b7fe63" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33563 } }
I20260925 00:42:52.793061 6521 raft_consensus.cc:388] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:42:52.793129 6521 raft_consensus.cc:482] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:42:52.793238 6521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:52.794270 6521 raft_consensus.cc:504] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04876fd00b144ce8a204ea9f0b7fe63" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33563 } }
I20260925 00:42:52.794632 6521 leader_election.cc:302] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a04876fd00b144ce8a204ea9f0b7fe63; no voters:
I20260925 00:42:52.795032 6521 leader_election.cc:288] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:42:52.795148 6526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:42:52.795372 6526 raft_consensus.cc:686] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [term 1 LEADER]: Becoming Leader. State: Replica: a04876fd00b144ce8a204ea9f0b7fe63, State: Running, Role: LEADER
I20260925 00:42:52.795688 6526 consensus_queue.cc:237] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04876fd00b144ce8a204ea9f0b7fe63" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33563 } }
I20260925 00:42:52.796453 6521 sys_catalog.cc:564] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:42:52.797402 6527 sys_catalog.cc:455] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a04876fd00b144ce8a204ea9f0b7fe63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04876fd00b144ce8a204ea9f0b7fe63" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33563 } } }
I20260925 00:42:52.797526 6527 sys_catalog.cc:458] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [sys.catalog]: This master's current role is: LEADER
I20260925 00:42:52.797891 6528 sys_catalog.cc:455] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a04876fd00b144ce8a204ea9f0b7fe63. Latest consensus state: current_term: 1 leader_uuid: "a04876fd00b144ce8a204ea9f0b7fe63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04876fd00b144ce8a204ea9f0b7fe63" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33563 } } }
I20260925 00:42:52.798038 6528 sys_catalog.cc:458] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63 [sys.catalog]: This master's current role is: LEADER
I20260925 00:42:52.799363 6535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 25 00:42:52 dist-test-slave-w8nh krb5kdc[6440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:52.802683 6535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:42:52.808995 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:52.782961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60758 (local address 127.6.72.126:33563)
0925 00:42:52.783491 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:52.783511 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:52.784646 (+ 1135us) server_negotiation.cc:407] Connection header received
0925 00:42:52.789611 (+ 4965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:52.789632 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:52.790016 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:52.790313 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:52.793560 (+ 3247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:52.794602 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:52.796319 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:52.797063 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:52.797255 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:52.802656 (+ 5401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:52.802664 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:52.802681 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:52.802719 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:52.804967 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:52.805667 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:52.805673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:52.805679 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:52.805839 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:52.806257 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:52.806262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:52.806264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:52.806822 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:52.807021 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:52.807810 (+ 789us) server_negotiation.cc:299] Negotiation successful
0925 00:42:52.808101 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":143,"threads_started":1}
I20260925 00:42:52.809911 6535 catalog_manager.cc:1347] Generated new cluster ID: 972fb3d62de143329fb7fb7478274fd1
I20260925 00:42:52.809978 6535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:42:52.820804 6535 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:42:52.822134 6535 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:42:52.837184 6535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a04876fd00b144ce8a204ea9f0b7fe63: Generated new TSK 0
I20260925 00:42:52.837988 6535 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:42:52.873492 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33563
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:53.002219 6547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:53.002528 6547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:53.002636 6547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:53.006842 6547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:53.006934 6547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:53.007023 6547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:42:53.012765 6547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33563
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:53.013892 6547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:53.015151 6547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:53.025075 6555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:53.025460 6553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:53.025492 6547 server_base.cc:1034] running on GCE node
W20260925 00:42:53.025534 6552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:53.025982 6547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:53.026669 6547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:53.027863 6547 hybrid_clock.cc:648] HybridClock initialized: now 1790296973027845 us; error 29 us; skew 500 ppm
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:53.030723 6547 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:42:53.031611 6547 webserver.cc:460] Webserver started at http://127.6.72.65:42357/ using document root <none> and password file <none>
I20260925 00:42:53.032186 6547 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:53.032335 6547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:53.032569 6547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:53.034619 6547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "7d73bb39f92e439dbe85ea8f766f6754"
format_stamp: "Formatted at 2026-09-25 00:42:53 on dist-test-slave-w8nh"
server_key: "95afafc793241ecb02e98ab25682c793"
server_key_iv: "8a7078bab87b72454f67ecea51308468"
server_key_version: "encryptionkey@0"
I20260925 00:42:53.035168 6547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "7d73bb39f92e439dbe85ea8f766f6754"
format_stamp: "Formatted at 2026-09-25 00:42:53 on dist-test-slave-w8nh"
server_key: "95afafc793241ecb02e98ab25682c793"
server_key_iv: "8a7078bab87b72454f67ecea51308468"
server_key_version: "encryptionkey@0"
I20260925 00:42:53.038949 6547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 00:42:53.041697 6562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:53.042999 6547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 00:42:53.043135 6547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "7d73bb39f92e439dbe85ea8f766f6754"
format_stamp: "Formatted at 2026-09-25 00:42:53 on dist-test-slave-w8nh"
server_key: "95afafc793241ecb02e98ab25682c793"
server_key_iv: "8a7078bab87b72454f67ecea51308468"
server_key_version: "encryptionkey@0"
I20260925 00:42:53.043254 6547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:53.057789 6547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:53.061496 6547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:53.061731 6547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:53.062389 6547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:42:53.063297 6547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:42:53.063380 6547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:53.063457 6547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:42:53.063511 6547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:53.072918 6547 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:35785
I20260925 00:42:53.072942 6675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:35785 every 8 connection(s)
I20260925 00:42:53.073872 6547 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:42:53.080941 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 6547
I20260925 00:42:53.081053 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:42:53.081348 6433 external_mini_cluster.cc:1428] Setting key bf8585edb90e34e128c3a0987ca8edb9
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:53.089933 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:53.075831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:38171 (local address 127.6.72.126:33563)
0925 00:42:53.075978 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:53.075983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:53.076989 (+ 1006us) server_negotiation.cc:407] Connection header received
0925 00:42:53.078392 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:53.078395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:53.078454 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:53.078558 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:53.079764 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.080375 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.081676 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.081915 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.082030 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:53.085400 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:53.085403 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:53.085405 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:53.085430 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:53.087278 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:53.087877 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:53.087883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:53.087887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:53.087987 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:53.088485 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:53.088491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:53.088494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:53.088790 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:53.088927 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:53.089551 (+ 624us) server_negotiation.cc:299] Negotiation successful
0925 00:42:53.089705 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 00:42:53.090788 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:53.076347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:33563 (local address 127.6.72.65:38171)
0925 00:42:53.076821 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:53.076874 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:53.078155 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:53.078737 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:53.078745 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:53.079127 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:53.079616 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:53.079629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.080500 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.080508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:53.081552 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:53.081561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.082071 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.082075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:53.082249 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:53.083124 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:53.083147 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:53.085202 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:53.087422 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:53.087427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:53.087441 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:53.087756 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:53.088112 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:53.088124 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:53.088126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:53.088373 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:53.088938 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:53.088944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:53.089254 (+ 310us) client_negotiation.cc:769] Sending connection context
0925 00:42:53.089443 (+ 189us) client_negotiation.cc:240] Negotiation successful
0925 00:42:53.089764 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:42:53.092398 6676 heartbeater.cc:347] Connected to a master server at 127.6.72.126:33563
I20260925 00:42:53.092680 6676 heartbeater.cc:464] Registering TS with master...
I20260925 00:42:53.093367 6676 heartbeater.cc:511] Master 127.6.72.126:33563 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:42:53.095193 6485 ts_manager.cc:194] Registered new tserver with Master: 7d73bb39f92e439dbe85ea8f766f6754 (127.6.72.65:35785)
I20260925 00:42:53.096520 6485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:38171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:42:53.111686 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33563
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:53.229904 6681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:53.230186 6681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:53.230242 6681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:53.234416 6681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:53.234480 6681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:53.234543 6681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:42:53.240039 6681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33563
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:53.241184 6681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:53.242177 6681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:53.252197 6689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:53.252286 6687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:53.252286 6686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:53.252529 6681 server_base.cc:1034] running on GCE node
I20260925 00:42:53.253211 6681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:53.253971 6681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:53.255232 6681 hybrid_clock.cc:648] HybridClock initialized: now 1790296973255185 us; error 31 us; skew 500 ppm
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:53.258126 6681 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:42:53.259132 6681 webserver.cc:460] Webserver started at http://127.6.72.66:34403/ using document root <none> and password file <none>
I20260925 00:42:53.259765 6681 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:53.259855 6681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:53.260092 6681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:53.262243 6681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "c3229928d8b6412a841878d30ac27ffe"
format_stamp: "Formatted at 2026-09-25 00:42:53 on dist-test-slave-w8nh"
server_key: "91d4fb8bc681c43ff7531b399e696fe0"
server_key_iv: "6d830dfe5d75cd3c927f615c88256a57"
server_key_version: "encryptionkey@0"
I20260925 00:42:53.262812 6681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "c3229928d8b6412a841878d30ac27ffe"
format_stamp: "Formatted at 2026-09-25 00:42:53 on dist-test-slave-w8nh"
server_key: "91d4fb8bc681c43ff7531b399e696fe0"
server_key_iv: "6d830dfe5d75cd3c927f615c88256a57"
server_key_version: "encryptionkey@0"
I20260925 00:42:53.267171 6681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 00:42:53.269922 6696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:53.271308 6681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 00:42:53.271448 6681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "c3229928d8b6412a841878d30ac27ffe"
format_stamp: "Formatted at 2026-09-25 00:42:53 on dist-test-slave-w8nh"
server_key: "91d4fb8bc681c43ff7531b399e696fe0"
server_key_iv: "6d830dfe5d75cd3c927f615c88256a57"
server_key_version: "encryptionkey@0"
I20260925 00:42:53.271574 6681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:53.302325 6681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:53.305685 6681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:53.305922 6681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:53.306551 6681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:42:53.307408 6681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:42:53.307480 6681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:53.307549 6681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:42:53.307605 6681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:53.317062 6681 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:45633
I20260925 00:42:53.317082 6809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:45633 every 8 connection(s)
I20260925 00:42:53.317970 6681 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:42:53.318646 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 6681
I20260925 00:42:53.318782 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:42:53.319037 6433 external_mini_cluster.cc:1428] Setting key bbfed1a1ecabee15dd793113b44345ca
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:42:53.332953 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:53.319959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:50981 (local address 127.6.72.126:33563)
0925 00:42:53.320087 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:53.320091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:53.321333 (+ 1242us) server_negotiation.cc:407] Connection header received
0925 00:42:53.322679 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:53.322682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:53.322739 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:53.322815 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:53.324099 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.324930 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.326107 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.326352 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.326472 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:53.329057 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:53.329060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:53.329062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:53.329088 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:53.330731 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:53.331337 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:53.331340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:53.331342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:53.331425 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:53.331765 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:53.331770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:53.331772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:53.331959 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:53.332065 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:53.332646 (+ 581us) server_negotiation.cc:299] Negotiation successful
0925 00:42:53.332773 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:42:53.333814 6812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:53.320639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:33563 (local address 127.6.72.66:50981)
0925 00:42:53.321160 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:53.321204 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:53.322458 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:53.323000 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:53.323007 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:53.323377 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:53.323929 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:53.323952 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.325063 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.325065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:53.325978 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:53.325987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.326516 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.326520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:53.326701 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:53.327336 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:53.327354 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:53.328889 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:53.330866 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:53.330872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:53.330887 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:53.331211 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:53.331517 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:53.331522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:53.331524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:53.331667 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:53.332064 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:53.332072 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:53.332408 (+ 336us) client_negotiation.cc:769] Sending connection context
0925 00:42:53.332607 (+ 199us) client_negotiation.cc:240] Negotiation successful
0925 00:42:53.332943 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":156,"threads_started":1}
I20260925 00:42:53.335117 6810 heartbeater.cc:347] Connected to a master server at 127.6.72.126:33563
I20260925 00:42:53.335387 6810 heartbeater.cc:464] Registering TS with master...
I20260925 00:42:53.335903 6810 heartbeater.cc:511] Master 127.6.72.126:33563 requested a full tablet report, sending...
I20260925 00:42:53.337198 6485 ts_manager.cc:194] Registered new tserver with Master: c3229928d8b6412a841878d30ac27ffe (127.6.72.66:45633)
I20260925 00:42:53.338068 6485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:50981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:42:53.347729 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33563
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:53.462509 6815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:53.462829 6815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:53.462910 6815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:53.467196 6815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:53.467289 6815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:53.467393 6815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:42:53.472945 6815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33563
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:53.474083 6815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:53.475225 6815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:53.485785 6820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:53.485785 6823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:53.485808 6821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:53.486096 6815 server_base.cc:1034] running on GCE node
I20260925 00:42:53.486555 6815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:53.487314 6815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:53.488499 6815 hybrid_clock.cc:648] HybridClock initialized: now 1790296973488482 us; error 28 us; skew 500 ppm
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:53.491418 6815 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:42:53.492501 6815 webserver.cc:460] Webserver started at http://127.6.72.67:44705/ using document root <none> and password file <none>
I20260925 00:42:53.493131 6815 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:53.493230 6815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:53.493495 6815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:53.495643 6815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "6be50a8ceab34ba7a87d376f221f4778"
format_stamp: "Formatted at 2026-09-25 00:42:53 on dist-test-slave-w8nh"
server_key: "684b416c174e65932d3e4b26d67dad66"
server_key_iv: "99554e4caa6cef2da183e9b1ab9b5ab5"
server_key_version: "encryptionkey@0"
I20260925 00:42:53.496335 6815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "6be50a8ceab34ba7a87d376f221f4778"
format_stamp: "Formatted at 2026-09-25 00:42:53 on dist-test-slave-w8nh"
server_key: "684b416c174e65932d3e4b26d67dad66"
server_key_iv: "99554e4caa6cef2da183e9b1ab9b5ab5"
server_key_version: "encryptionkey@0"
I20260925 00:42:53.500635 6815 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 00:42:53.503275 6830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:53.504662 6815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:42:53.504812 6815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "6be50a8ceab34ba7a87d376f221f4778"
format_stamp: "Formatted at 2026-09-25 00:42:53 on dist-test-slave-w8nh"
server_key: "684b416c174e65932d3e4b26d67dad66"
server_key_iv: "99554e4caa6cef2da183e9b1ab9b5ab5"
server_key_version: "encryptionkey@0"
I20260925 00:42:53.504935 6815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:53.536849 6815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:53.540499 6815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:53.540740 6815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:53.541409 6815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:42:53.542270 6815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:42:53.542342 6815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:53.542415 6815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:42:53.542457 6815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:53.552794 6815 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:36323
I20260925 00:42:53.552817 6943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:36323 every 8 connection(s)
I20260925 00:42:53.553755 6815 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:42:53.554652 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 6815
I20260925 00:42:53.554797 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:42:53.555068 6433 external_mini_cluster.cc:1428] Setting key 42616b463d644fb90714610cfc57874c
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:53.568776 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:53.555928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:41749 (local address 127.6.72.126:33563)
0925 00:42:53.556059 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:53.556064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:53.557257 (+ 1193us) server_negotiation.cc:407] Connection header received
0925 00:42:53.558781 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:53.558784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:53.558839 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:53.558915 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:53.560112 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.560720 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.561919 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.562147 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.562264 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:53.564795 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:53.564799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:53.564801 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:53.564832 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:53.566316 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:53.567022 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:53.567027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:53.567028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:53.567113 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:53.567468 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:53.567473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:53.567475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:53.567662 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:53.567772 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:53.568416 (+ 644us) server_negotiation.cc:299] Negotiation successful
0925 00:42:53.568570 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:42:53.569742 6946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:53.556565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:33563 (local address 127.6.72.67:41749)
0925 00:42:53.557100 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:53.557150 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:53.558568 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:53.559091 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:53.559098 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:53.559485 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:53.559970 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:53.559984 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.560846 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.560849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:53.561771 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:53.561780 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.562301 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.562304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:53.562473 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:53.563036 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:53.563052 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:53.564627 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:53.566532 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:53.566538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:53.566554 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:53.566916 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:53.567216 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:53.567220 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:53.567222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:53.567370 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:53.567769 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:53.567775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:53.568065 (+ 290us) client_negotiation.cc:769] Sending connection context
0925 00:42:53.568303 (+ 238us) client_negotiation.cc:240] Negotiation successful
0925 00:42:53.568678 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":169,"threads_started":1}
I20260925 00:42:53.570906 6944 heartbeater.cc:347] Connected to a master server at 127.6.72.126:33563
I20260925 00:42:53.571168 6944 heartbeater.cc:464] Registering TS with master...
I20260925 00:42:53.571689 6944 heartbeater.cc:511] Master 127.6.72.126:33563 requested a full tablet report, sending...
I20260925 00:42:53.572779 6485 ts_manager.cc:194] Registered new tserver with Master: 6be50a8ceab34ba7a87d376f221f4778 (127.6.72.67:36323)
I20260925 00:42:53.573366 6485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:41749
I20260925 00:42:53.582598 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296973, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296973, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:42:53.610754 6951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:53.599985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.6.72.65:35785)
0925 00:42:53.600371 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:53.600377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:53.600402 (+ 25us) server_negotiation.cc:407] Connection header received
0925 00:42:53.600518 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:53.600523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:53.600726 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:53.600854 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:53.601755 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.602551 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.603473 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.603880 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.603975 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:53.606832 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:53.606844 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:53.606860 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:53.606887 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:53.608703 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:53.609213 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:53.609219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:53.609220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:53.609346 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:53.609658 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:53.609668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:53.609670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:53.609985 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:53.610230 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:53.610407 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 00:42:53.610568 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":107,"threads_started":1}
W20260925 00:42:53.611783 6590 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42516
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:42:53.639618 6951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:53.630743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42524 (local address 127.6.72.65:35785)
0925 00:42:53.630897 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:53.630901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:53.630920 (+ 19us) server_negotiation.cc:407] Connection header received
0925 00:42:53.630995 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:53.630998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:53.631038 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:53.631128 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:53.631885 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.632494 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.633376 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:53.633601 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:53.633715 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:53.636459 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:53.636463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:53.636487 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:53.636524 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:53.637918 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:53.638371 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:53.638375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:53.638377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:53.638466 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:53.638752 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:53.638756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:53.638758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:53.638943 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:53.639106 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:53.639327 (+ 221us) server_negotiation.cc:299] Negotiation successful
0925 00:42:53.639450 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 00:42:53.640733 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 6547
I20260925 00:42:53.647414 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 6681
I20260925 00:42:53.654887 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 6815
I20260925 00:42:53.660104 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 6453
2026-09-25T00:42:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2297 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6958](info): set up 2 sockets
Sep 25 00:42:53 dist-test-slave-w8nh krb5kdc[6958](info): commencing operation
krb5kdc: starting...
W20260925 00:42:54.708802 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:42:54 dist-test-slave-w8nh krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:42:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:42:54.836148 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:37495
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:40283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:37495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:54.950089 6971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:54.950383 6971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:54.950425 6971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:54.954483 6971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:42:54.954552 6971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:54.954571 6971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:54.954589 6971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:42:54.954604 6971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:42:54.959977 6971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:37495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:37495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:54.961251 6971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:54.962296 6971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:54.971282 6979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:54.971385 6977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:54.971316 6976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:54.971737 6971 server_base.cc:1034] running on GCE node
I20260925 00:42:54.972384 6971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:54.973562 6971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:54.974759 6971 hybrid_clock.cc:648] HybridClock initialized: now 1790296974974744 us; error 37 us; skew 500 ppm
Sep 25 00:42:54 dist-test-slave-w8nh krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296974, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:54.977609 6971 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:42:54.978575 6971 webserver.cc:460] Webserver started at http://127.6.72.126:44405/ using document root <none> and password file <none>
I20260925 00:42:54.979162 6971 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:54.979252 6971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:54.979493 6971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:54.981698 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "fd00d759ac494485b505c89548a70df8"
format_stamp: "Formatted at 2026-09-25 00:42:54 on dist-test-slave-w8nh"
server_key: "06d583b04faae9e1ca0c96d1ec2dbc46"
server_key_iv: "b6b8e8c4161c5ba16057fffd0d63a61e"
server_key_version: "encryptionkey@0"
I20260925 00:42:54.982280 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "fd00d759ac494485b505c89548a70df8"
format_stamp: "Formatted at 2026-09-25 00:42:54 on dist-test-slave-w8nh"
server_key: "06d583b04faae9e1ca0c96d1ec2dbc46"
server_key_iv: "b6b8e8c4161c5ba16057fffd0d63a61e"
server_key_version: "encryptionkey@0"
I20260925 00:42:54.986135 6971 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260925 00:42:54.988652 6986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:54.989882 6971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:42:54.990036 6971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "fd00d759ac494485b505c89548a70df8"
format_stamp: "Formatted at 2026-09-25 00:42:54 on dist-test-slave-w8nh"
server_key: "06d583b04faae9e1ca0c96d1ec2dbc46"
server_key_iv: "b6b8e8c4161c5ba16057fffd0d63a61e"
server_key_version: "encryptionkey@0"
I20260925 00:42:54.990159 6971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:55.020206 6971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:55.023622 6971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:55.023850 6971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:55.030743 6971 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:37495
I20260925 00:42:55.030831 7038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:37495 every 8 connection(s)
I20260925 00:42:55.031790 6971 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:42:55.032840 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 6971
I20260925 00:42:55.032949 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:42:55.033210 6433 external_mini_cluster.cc:1428] Setting key 2cffa99a6580c3cbe026bcfbc607966c
I20260925 00:42:55.035351 7039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:55.040946 7039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8: Bootstrap starting.
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:55.043365 7039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:55.044296 7039 log.cc:824] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8: Log is configured to *not* fsync() on all Append() calls
I20260925 00:42:55.046731 7039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8: No bootstrap required, opened a new log
I20260925 00:42:55.048100 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.034651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35544 (local address 127.6.72.126:37495)
0925 00:42:55.035131 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.035150 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.035194 (+ 44us) server_negotiation.cc:407] Connection header received
0925 00:42:55.035933 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.035951 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.036363 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.036706 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:55.037505 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.038483 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.039337 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.039821 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.039990 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.042649 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:55.042655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:55.042668 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:55.042696 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:55.044935 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.045582 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.045592 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.045597 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.045742 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.046048 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.046050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.046052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.046453 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:55.046619 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.046967 (+ 348us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.047316 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":150,"threads_started":1}
I20260925 00:42:55.050443 7039 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd00d759ac494485b505c89548a70df8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37495 } }
I20260925 00:42:55.050665 7039 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:55.050751 7039 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd00d759ac494485b505c89548a70df8, State: Initialized, Role: FOLLOWER
I20260925 00:42:55.051172 7039 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd00d759ac494485b505c89548a70df8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37495 } }
I20260925 00:42:55.051323 7039 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:42:55.051390 7039 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:42:55.051487 7039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:55.052551 7039 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd00d759ac494485b505c89548a70df8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37495 } }
I20260925 00:42:55.052912 7039 leader_election.cc:302] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd00d759ac494485b505c89548a70df8; no voters:
I20260925 00:42:55.053257 7039 leader_election.cc:288] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:42:55.053427 7044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:42:55.053682 7044 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [term 1 LEADER]: Becoming Leader. State: Replica: fd00d759ac494485b505c89548a70df8, State: Running, Role: LEADER
I20260925 00:42:55.054001 7044 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd00d759ac494485b505c89548a70df8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37495 } }
I20260925 00:42:55.054529 7039 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:42:55.055650 7045 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd00d759ac494485b505c89548a70df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd00d759ac494485b505c89548a70df8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37495 } } }
I20260925 00:42:55.055769 7045 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [sys.catalog]: This master's current role is: LEADER
I20260925 00:42:55.056447 7046 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd00d759ac494485b505c89548a70df8. Latest consensus state: current_term: 1 leader_uuid: "fd00d759ac494485b505c89548a70df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd00d759ac494485b505c89548a70df8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37495 } } }
I20260925 00:42:55.056598 7046 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8 [sys.catalog]: This master's current role is: LEADER
W20260925 00:42:55.059227 7059 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:42:55.059319 7059 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:42:55.059389 7053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:42:55.060053 7053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:42:55.067255 7053 catalog_manager.cc:1347] Generated new cluster ID: dde8a3cb8a4248a79a91027bb7f6716c
I20260925 00:42:55.067358 7053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:42:55.078300 7053 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:42:55.079658 7053 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:42:55.092815 7053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd00d759ac494485b505c89548a70df8: Generated new TSK 0
I20260925 00:42:55.093631 7053 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:42:55.132608 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:37495
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:40283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:55.251514 7065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:55.251827 7065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:55.251874 7065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:55.256524 7065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:55.256599 7065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:55.256722 7065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:42:55.262519 7065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:37495
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:55.263692 7065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:55.264786 7065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:55.275071 7071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:55.275058 7070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:55.275058 7073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:55.275484 7065 server_base.cc:1034] running on GCE node
I20260925 00:42:55.275920 7065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:55.276656 7065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:55.277865 7065 hybrid_clock.cc:648] HybridClock initialized: now 1790296975277849 us; error 37 us; skew 500 ppm
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:55.280789 7065 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:42:55.281759 7065 webserver.cc:460] Webserver started at http://127.6.72.65:39047/ using document root <none> and password file <none>
I20260925 00:42:55.282375 7065 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:55.282438 7065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:55.282668 7065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:55.284863 7065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "aa07792917bb4ac6b56ff5606093e8a8"
format_stamp: "Formatted at 2026-09-25 00:42:55 on dist-test-slave-w8nh"
server_key: "14b2ef8b16c5d6c8aa7ba756c34e427f"
server_key_iv: "cda1bdc8e671179658bf8acf2bd927bc"
server_key_version: "encryptionkey@0"
I20260925 00:42:55.285449 7065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "aa07792917bb4ac6b56ff5606093e8a8"
format_stamp: "Formatted at 2026-09-25 00:42:55 on dist-test-slave-w8nh"
server_key: "14b2ef8b16c5d6c8aa7ba756c34e427f"
server_key_iv: "cda1bdc8e671179658bf8acf2bd927bc"
server_key_version: "encryptionkey@0"
I20260925 00:42:55.289498 7065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:42:55.292375 7080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:55.294046 7065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:42:55.294221 7065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "aa07792917bb4ac6b56ff5606093e8a8"
format_stamp: "Formatted at 2026-09-25 00:42:55 on dist-test-slave-w8nh"
server_key: "14b2ef8b16c5d6c8aa7ba756c34e427f"
server_key_iv: "cda1bdc8e671179658bf8acf2bd927bc"
server_key_version: "encryptionkey@0"
I20260925 00:42:55.294358 7065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:55.310192 7065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:55.313531 7065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:55.313779 7065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:55.314481 7065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:42:55.315398 7065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:42:55.315443 7065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:55.315485 7065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:42:55.315500 7065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:55.324894 7065 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:35673
I20260925 00:42:55.324915 7193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:35673 every 8 connection(s)
I20260925 00:42:55.325845 7065 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:42:55.331499 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 7065
I20260925 00:42:55.331630 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:42:55.331909 6433 external_mini_cluster.cc:1428] Setting key 3e98c5a13ceffce280518d7ce9646855
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:55.341951 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.327652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:47049 (local address 127.6.72.126:37495)
0925 00:42:55.327814 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.327819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.328854 (+ 1035us) server_negotiation.cc:407] Connection header received
0925 00:42:55.330159 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.330162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.330212 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.330295 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:55.331698 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.332397 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.333886 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.334118 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.334235 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.337261 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:55.337268 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:55.337274 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:55.337310 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:55.339262 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.339839 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.339844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.339847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.339942 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.340485 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.340490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.340494 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.340773 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:55.340913 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.341538 (+ 625us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.341688 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:42:55.342725 7196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.328145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:37495 (local address 127.6.72.65:47049)
0925 00:42:55.328682 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:55.328730 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:55.329954 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:55.330466 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:55.330473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:55.330857 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:55.331513 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.331535 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.332575 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.332579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.333755 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.333768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.334300 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.334308 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.334550 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.335290 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:55.335313 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:55.337066 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:55.339402 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:55.339407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:55.339422 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:55.339726 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:55.340079 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:55.340082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:55.340083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:55.340376 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:55.340936 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:55.340943 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:55.341227 (+ 284us) client_negotiation.cc:769] Sending connection context
0925 00:42:55.341427 (+ 200us) client_negotiation.cc:240] Negotiation successful
0925 00:42:55.341721 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":116,"threads_started":1}
I20260925 00:42:55.344175 7194 heartbeater.cc:347] Connected to a master server at 127.6.72.126:37495
I20260925 00:42:55.344511 7194 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:42:55.345207 7194 heartbeater.cc:511] Master 127.6.72.126:37495 requested a full tablet report, sending...
I20260925 00:42:55.346933 7003 ts_manager.cc:194] Registered new tserver with Master: aa07792917bb4ac6b56ff5606093e8a8 (127.6.72.65:35673)
I20260925 00:42:55.348318 7003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:47049
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:42:55.363583 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:37495
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:40283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:55.478727 7199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:55.479206 7199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:55.479287 7199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:55.483632 7199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:55.483697 7199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:55.483767 7199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:42:55.489212 7199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:37495
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:55.490362 7199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:55.491354 7199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:55.501855 7205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:55.501848 7207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:55.501848 7204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:55.502182 7199 server_base.cc:1034] running on GCE node
I20260925 00:42:55.502643 7199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:55.503315 7199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:55.504477 7199 hybrid_clock.cc:648] HybridClock initialized: now 1790296975504460 us; error 28 us; skew 500 ppm
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:55.523286 7199 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:42:55.524483 7199 webserver.cc:460] Webserver started at http://127.6.72.66:38787/ using document root <none> and password file <none>
I20260925 00:42:55.525131 7199 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:55.525242 7199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:55.525498 7199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:55.527776 7199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "2dcd1054f86c4b06b5e909da54bec822"
format_stamp: "Formatted at 2026-09-25 00:42:55 on dist-test-slave-w8nh"
server_key: "1f357aca29c6df3e882ca44a139291e4"
server_key_iv: "8a9cb590daea4ab3eaa0b9a27b855905"
server_key_version: "encryptionkey@0"
I20260925 00:42:55.528476 7199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "2dcd1054f86c4b06b5e909da54bec822"
format_stamp: "Formatted at 2026-09-25 00:42:55 on dist-test-slave-w8nh"
server_key: "1f357aca29c6df3e882ca44a139291e4"
server_key_iv: "8a9cb590daea4ab3eaa0b9a27b855905"
server_key_version: "encryptionkey@0"
I20260925 00:42:55.532500 7199 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:42:55.535075 7214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:55.536500 7199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:42:55.536669 7199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "2dcd1054f86c4b06b5e909da54bec822"
format_stamp: "Formatted at 2026-09-25 00:42:55 on dist-test-slave-w8nh"
server_key: "1f357aca29c6df3e882ca44a139291e4"
server_key_iv: "8a9cb590daea4ab3eaa0b9a27b855905"
server_key_version: "encryptionkey@0"
I20260925 00:42:55.536798 7199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:55.549345 7199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:55.552899 7199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:55.553130 7199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:55.553761 7199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:42:55.554656 7199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:42:55.554764 7199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:55.554828 7199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:42:55.554867 7199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:55.564309 7199 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:43189
I20260925 00:42:55.564327 7327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:43189 every 8 connection(s)
I20260925 00:42:55.565303 7199 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:42:55.571064 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 7199
I20260925 00:42:55.571161 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:42:55.571449 6433 external_mini_cluster.cc:1428] Setting key 351f50e003ecf514a2068e6039b8bbce
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:42:55.582078 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.567329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:46279 (local address 127.6.72.126:37495)
0925 00:42:55.567492 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.567496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.568509 (+ 1013us) server_negotiation.cc:407] Connection header received
0925 00:42:55.569867 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.569870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.569917 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.569994 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:55.571328 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.572038 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.573545 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.573798 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.573932 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.577059 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:55.577066 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:55.577072 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:55.577109 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:55.579134 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.579899 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.579905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.579908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.580011 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.580497 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.580500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.580502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.580793 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:55.580917 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.581718 (+ 801us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.581870 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 00:42:55.583057 7330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.567697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:37495 (local address 127.6.72.66:46279)
0925 00:42:55.568330 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:55.568393 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:55.569663 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:55.570205 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:55.570212 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:55.570591 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:55.571147 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.571169 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.572189 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.572193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.573406 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.573416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.573977 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.573982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.574178 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.574876 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:55.574894 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:55.576801 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:55.579281 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:55.579287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:55.579440 (+ 153us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:55.579766 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:55.580111 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:55.580114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:55.580115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:55.580381 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:55.580925 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:55.580931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:55.581289 (+ 358us) client_negotiation.cc:769] Sending connection context
0925 00:42:55.581660 (+ 371us) client_negotiation.cc:240] Negotiation successful
0925 00:42:55.582051 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:42:55.584478 7328 heartbeater.cc:347] Connected to a master server at 127.6.72.126:37495
I20260925 00:42:55.584851 7328 heartbeater.cc:464] Registering TS with master...
I20260925 00:42:55.585417 7328 heartbeater.cc:511] Master 127.6.72.126:37495 requested a full tablet report, sending...
I20260925 00:42:55.586606 7003 ts_manager.cc:194] Registered new tserver with Master: 2dcd1054f86c4b06b5e909da54bec822 (127.6.72.66:43189)
I20260925 00:42:55.587241 7003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:46279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:42:55.603798 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:37495
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:40283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:55.718838 7333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:55.719146 7333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:55.719200 7333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:55.723518 7333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:55.723601 7333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:55.723666 7333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:42:55.729210 7333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:37495
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:55.730443 7333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:55.731467 7333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:55.741688 7338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:55.741734 7333 server_base.cc:1034] running on GCE node
W20260925 00:42:55.741712 7341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:55.741693 7339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:55.742417 7333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:55.743181 7333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:55.744328 7333 hybrid_clock.cc:648] HybridClock initialized: now 1790296975744281 us; error 53 us; skew 500 ppm
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:55.747197 7333 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:42:55.748116 7333 webserver.cc:460] Webserver started at http://127.6.72.67:35339/ using document root <none> and password file <none>
I20260925 00:42:55.748775 7333 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:55.748873 7333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:55.749125 7333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:55.751283 7333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "3d7e500b1b82446180190c6a2a15be3b"
format_stamp: "Formatted at 2026-09-25 00:42:55 on dist-test-slave-w8nh"
server_key: "8fafd0f9da3dfbb68693b218e65b1f68"
server_key_iv: "954e62fecaf6a7101600cbcd5c63e675"
server_key_version: "encryptionkey@0"
I20260925 00:42:55.751881 7333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "3d7e500b1b82446180190c6a2a15be3b"
format_stamp: "Formatted at 2026-09-25 00:42:55 on dist-test-slave-w8nh"
server_key: "8fafd0f9da3dfbb68693b218e65b1f68"
server_key_iv: "954e62fecaf6a7101600cbcd5c63e675"
server_key_version: "encryptionkey@0"
I20260925 00:42:55.755956 7333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 00:42:55.758682 7348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:55.760404 7333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:42:55.760531 7333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "3d7e500b1b82446180190c6a2a15be3b"
format_stamp: "Formatted at 2026-09-25 00:42:55 on dist-test-slave-w8nh"
server_key: "8fafd0f9da3dfbb68693b218e65b1f68"
server_key_iv: "954e62fecaf6a7101600cbcd5c63e675"
server_key_version: "encryptionkey@0"
I20260925 00:42:55.760664 7333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:55.785953 7333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:55.789565 7333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:55.789780 7333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:55.790427 7333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:42:55.791304 7333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:42:55.791360 7333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:55.791430 7333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:42:55.791469 7333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:55.800618 7333 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:36171
I20260925 00:42:55.800643 7461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:36171 every 8 connection(s)
I20260925 00:42:55.801657 7333 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:42:55.811172 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 7333
I20260925 00:42:55.811319 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:42:55.811637 6433 external_mini_cluster.cc:1428] Setting key a585fad3f017d19cacb99832cc713542
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:55.816567 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.803572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:33319 (local address 127.6.72.126:37495)
0925 00:42:55.803721 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.803725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.804716 (+ 991us) server_negotiation.cc:407] Connection header received
0925 00:42:55.806145 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.806148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.806209 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.806290 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:55.807542 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.808096 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.809360 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.809594 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.809746 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.812722 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:55.812726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:55.812733 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:55.812766 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:55.814347 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.814940 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.814943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.814945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.815038 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.815474 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.815476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.815478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.815676 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:55.815753 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.816327 (+ 574us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.816445 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 00:42:55.817524 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.803982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:37495 (local address 127.6.72.67:33319)
0925 00:42:55.804532 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:55.804577 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:55.805916 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:55.806472 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:55.806480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:55.806869 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:55.807378 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.807390 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.808294 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.808298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.809201 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.809220 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.809781 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.809785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.809968 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.810636 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:55.810653 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:55.812498 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:55.814500 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:55.814505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:55.814520 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:55.814805 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:55.815130 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:55.815132 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:55.815133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:55.815312 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:55.815772 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:55.815777 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:55.816039 (+ 262us) client_negotiation.cc:769] Sending connection context
0925 00:42:55.816230 (+ 191us) client_negotiation.cc:240] Negotiation successful
0925 00:42:55.816596 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":121,"threads_started":1}
I20260925 00:42:55.818672 7462 heartbeater.cc:347] Connected to a master server at 127.6.72.126:37495
I20260925 00:42:55.818974 7462 heartbeater.cc:464] Registering TS with master...
I20260925 00:42:55.819547 7462 heartbeater.cc:511] Master 127.6.72.126:37495 requested a full tablet report, sending...
I20260925 00:42:55.820752 7003 ts_manager.cc:194] Registered new tserver with Master: 3d7e500b1b82446180190c6a2a15be3b (127.6.72.67:36171)
I20260925 00:42:55.821357 7003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:33319
I20260925 00:42:55.826113 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:42:55.837018 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.829839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35560 (local address 127.6.72.126:37495)
0925 00:42:55.829972 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.829976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.830091 (+ 115us) server_negotiation.cc:407] Connection header received
0925 00:42:55.830213 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.830215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.830257 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.830332 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:55.831026 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.831551 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.832408 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.832607 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.832686 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.834101 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:55.834104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:55.834106 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:55.834127 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:55.835463 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.835869 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.835872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.835873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.835944 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.836167 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.836169 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.836170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.836396 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:55.836496 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.836798 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.836895 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 00:42:55.842581 7003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:42:55.845886 7003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:42:55.861987 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.855955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.6.72.67:36171)
0925 00:42:55.856430 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.856443 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.856467 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:42:55.857114 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.857121 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.857367 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.857520 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:55.858013 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.858830 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.860373 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.861276 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.861394 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.861550 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.861727 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.861833 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":120,"threads_started":1}
I20260925 00:42:55.862139 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.856658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:43189 (local address 127.0.0.1:34474)
0925 00:42:55.856874 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:55.856888 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:55.856969 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:55.857967 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:55.857972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:55.857992 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:55.858073 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.858079 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.859482 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.859485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.860651 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.860659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.861679 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.861683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.861780 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.861806 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 00:42:55.861855 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 00:42:55.861930 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":61,"thread_start_us":91,"threads_started":1}
I20260925 00:42:55.861984 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.855841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:36171 (local address 127.0.0.1:33304)
0925 00:42:55.856224 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:55.856326 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:55.856650 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:55.857718 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:55.857723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:55.857745 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:55.857865 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.857870 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.858954 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.858960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.860217 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.860226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.861403 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.861406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.861509 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.861545 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 00:42:55.861684 (+ 139us) client_negotiation.cc:240] Negotiation successful
0925 00:42:55.861817 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260925 00:42:55.862401 7477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.856991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35673 (local address 127.0.0.1:37178)
0925 00:42:55.857245 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:55.857258 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:55.857374 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:55.858167 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:55.858170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:55.858187 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:55.858281 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.858298 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.859732 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.859735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.860981 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.860991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.862066 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.862069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.862146 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.862165 (+ 19us) client_negotiation.cc:769] Sending connection context
0925 00:42:55.862215 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 00:42:55.862278 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":53,"thread_start_us":94,"threads_started":1}
I20260925 00:42:55.862496 7476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.856850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34474 (local address 127.6.72.66:43189)
0925 00:42:55.857495 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.857508 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.857533 (+ 25us) server_negotiation.cc:407] Connection header received
0925 00:42:55.857619 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.857623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.857825 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.857973 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:55.858181 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.859325 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.860769 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.861537 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.862059 (+ 522us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.862197 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.862247 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.862345 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":128,"threads_started":1}
I20260925 00:42:55.863615 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.857434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37178 (local address 127.6.72.65:35673)
0925 00:42:55.857755 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.857761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.857781 (+ 20us) server_negotiation.cc:407] Connection header received
0925 00:42:55.857853 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.857858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.858022 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.858435 (+ 413us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:55.858561 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.859618 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.861112 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.861938 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.862596 (+ 658us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.862897 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.863374 (+ 477us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.863475 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":123,"threads_started":1}
I20260925 00:42:55.865658 7396 tablet_service.cc:1467] Processing CreateTablet for tablet 6c012ab364b246e28bcafcf58da17cf2 (DEFAULT_TABLE table=test-table [id=6a8cca1d6c374185bbe5d977b39bec42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:42:55.865691 7262 tablet_service.cc:1467] Processing CreateTablet for tablet 6c012ab364b246e28bcafcf58da17cf2 (DEFAULT_TABLE table=test-table [id=6a8cca1d6c374185bbe5d977b39bec42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:42:55.866027 7128 tablet_service.cc:1467] Processing CreateTablet for tablet 6c012ab364b246e28bcafcf58da17cf2 (DEFAULT_TABLE table=test-table [id=6a8cca1d6c374185bbe5d977b39bec42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:42:55.866748 7396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c012ab364b246e28bcafcf58da17cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:55.866746 7262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c012ab364b246e28bcafcf58da17cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:55.866848 7128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c012ab364b246e28bcafcf58da17cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:55.873229 7479 tablet_bootstrap.cc:492] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b: Bootstrap starting.
I20260925 00:42:55.874497 7481 tablet_bootstrap.cc:492] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8: Bootstrap starting.
I20260925 00:42:55.875505 7479 tablet_bootstrap.cc:654] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:55.876327 7481 tablet_bootstrap.cc:654] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:55.876601 7480 tablet_bootstrap.cc:492] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822: Bootstrap starting.
I20260925 00:42:55.877007 7479 log.cc:824] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b: Log is configured to *not* fsync() on all Append() calls
I20260925 00:42:55.877077 7481 log.cc:824] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8: Log is configured to *not* fsync() on all Append() calls
I20260925 00:42:55.879041 7480 tablet_bootstrap.cc:654] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:55.879242 7479 tablet_bootstrap.cc:492] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b: No bootstrap required, opened a new log
I20260925 00:42:55.879381 7481 tablet_bootstrap.cc:492] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8: No bootstrap required, opened a new log
I20260925 00:42:55.879460 7479 ts_tablet_manager.cc:1397] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:42:55.879534 7481 ts_tablet_manager.cc:1397] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 00:42:55.880412 7480 log.cc:824] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822: Log is configured to *not* fsync() on all Append() calls
I20260925 00:42:55.882176 7480 tablet_bootstrap.cc:492] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822: No bootstrap required, opened a new log
I20260925 00:42:55.882411 7480 ts_tablet_manager.cc:1397] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260925 00:42:55.884142 7481 raft_consensus.cc:348] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7e500b1b82446180190c6a2a15be3b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36171 } } peers { permanent_uuid: "2dcd1054f86c4b06b5e909da54bec822" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43189 } } peers { permanent_uuid: "aa07792917bb4ac6b56ff5606093e8a8" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35673 } }
I20260925 00:42:55.884142 7479 raft_consensus.cc:348] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7e500b1b82446180190c6a2a15be3b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36171 } } peers { permanent_uuid: "2dcd1054f86c4b06b5e909da54bec822" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43189 } } peers { permanent_uuid: "aa07792917bb4ac6b56ff5606093e8a8" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35673 } }
I20260925 00:42:55.884477 7481 raft_consensus.cc:374] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:55.884477 7479 raft_consensus.cc:374] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:55.884538 7481 raft_consensus.cc:729] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa07792917bb4ac6b56ff5606093e8a8, State: Initialized, Role: FOLLOWER
I20260925 00:42:55.884538 7479 raft_consensus.cc:729] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7e500b1b82446180190c6a2a15be3b, State: Initialized, Role: FOLLOWER
I20260925 00:42:55.885032 7481 consensus_queue.cc:260] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7e500b1b82446180190c6a2a15be3b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36171 } } peers { permanent_uuid: "2dcd1054f86c4b06b5e909da54bec822" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43189 } } peers { permanent_uuid: "aa07792917bb4ac6b56ff5606093e8a8" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35673 } }
I20260925 00:42:55.885032 7479 consensus_queue.cc:260] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7e500b1b82446180190c6a2a15be3b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36171 } } peers { permanent_uuid: "2dcd1054f86c4b06b5e909da54bec822" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43189 } } peers { permanent_uuid: "aa07792917bb4ac6b56ff5606093e8a8" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35673 } }
I20260925 00:42:55.886000 7462 heartbeater.cc:503] Master 127.6.72.126:37495 was elected leader, sending a full tablet report...
I20260925 00:42:55.886144 7481 ts_tablet_manager.cc:1428] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260925 00:42:55.886144 7479 ts_tablet_manager.cc:1428] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260925 00:42:55.886214 7480 raft_consensus.cc:348] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7e500b1b82446180190c6a2a15be3b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36171 } } peers { permanent_uuid: "2dcd1054f86c4b06b5e909da54bec822" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43189 } } peers { permanent_uuid: "aa07792917bb4ac6b56ff5606093e8a8" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35673 } }
I20260925 00:42:55.886528 7480 raft_consensus.cc:374] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:55.886583 7480 raft_consensus.cc:729] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dcd1054f86c4b06b5e909da54bec822, State: Initialized, Role: FOLLOWER
I20260925 00:42:55.887056 7480 consensus_queue.cc:260] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7e500b1b82446180190c6a2a15be3b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36171 } } peers { permanent_uuid: "2dcd1054f86c4b06b5e909da54bec822" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43189 } } peers { permanent_uuid: "aa07792917bb4ac6b56ff5606093e8a8" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35673 } }
I20260925 00:42:55.887248 7194 heartbeater.cc:503] Master 127.6.72.126:37495 was elected leader, sending a full tablet report...
I20260925 00:42:55.887753 7328 heartbeater.cc:503] Master 127.6.72.126:37495 was elected leader, sending a full tablet report...
I20260925 00:42:55.888149 7480 ts_tablet_manager.cc:1428] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260925 00:42:55.891211 7487 raft_consensus.cc:482] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:42:55.891403 7487 raft_consensus.cc:504] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7e500b1b82446180190c6a2a15be3b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36171 } } peers { permanent_uuid: "2dcd1054f86c4b06b5e909da54bec822" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43189 } } peers { permanent_uuid: "aa07792917bb4ac6b56ff5606093e8a8" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35673 } }
I20260925 00:42:55.892663 7487 leader_election.cc:288] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d7e500b1b82446180190c6a2a15be3b (127.6.72.67:36171), aa07792917bb4ac6b56ff5606093e8a8 (127.6.72.65:35673)
I20260925 00:42:55.896579 7330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.892894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:36171 (local address 127.6.72.66:52347)
0925 00:42:55.893025 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:55.893041 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:55.893125 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:55.893755 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:55.893758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:55.893775 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:55.893896 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.893901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.894644 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.894647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.895437 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.895445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.896122 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.896126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.896226 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.896273 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 00:42:55.896377 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 00:42:55.896453 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 00:42:55.896579 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.892921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:52347 (local address 127.6.72.67:36171)
0925 00:42:55.893528 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.893534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.893552 (+ 18us) server_negotiation.cc:407] Connection header received
0925 00:42:55.893596 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.893599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.893645 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.893731 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:55.894001 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.894520 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.895559 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.895987 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.896120 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.896184 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.896400 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.896462 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511}
I20260925 00:42:55.896790 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.893322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:38723 (local address 127.6.72.65:35673)
0925 00:42:55.893437 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.893441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.893678 (+ 237us) server_negotiation.cc:407] Connection header received
0925 00:42:55.893816 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.893819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.893862 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.893954 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:55.894200 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.894785 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.895802 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.896220 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.896375 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.896437 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.896614 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.896682 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 00:42:55.896801 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.893184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35673 (local address 127.6.72.66:38723)
0925 00:42:55.893583 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:55.893594 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:55.893727 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:55.893962 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:55.893965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:55.893979 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:55.894093 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.894098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.894916 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.894920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.895682 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:55.895689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.896377 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.896380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:55.896482 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.896502 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 00:42:55.896599 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 00:42:55.896667 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":138,"threads_started":1}
I20260925 00:42:55.897367 7148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c012ab364b246e28bcafcf58da17cf2" candidate_uuid: "2dcd1054f86c4b06b5e909da54bec822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa07792917bb4ac6b56ff5606093e8a8" is_pre_election: true
I20260925 00:42:55.897445 7416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c012ab364b246e28bcafcf58da17cf2" candidate_uuid: "2dcd1054f86c4b06b5e909da54bec822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d7e500b1b82446180190c6a2a15be3b" is_pre_election: true
I20260925 00:42:55.897679 7148 raft_consensus.cc:2454] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dcd1054f86c4b06b5e909da54bec822 in term 0.
I20260925 00:42:55.897781 7416 raft_consensus.cc:2454] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dcd1054f86c4b06b5e909da54bec822 in term 0.
I20260925 00:42:55.898231 7217 leader_election.cc:302] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dcd1054f86c4b06b5e909da54bec822, 3d7e500b1b82446180190c6a2a15be3b; no voters:
I20260925 00:42:55.898531 7487 raft_consensus.cc:2781] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:42:55.898638 7487 raft_consensus.cc:482] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:42:55.898684 7487 raft_consensus.cc:3037] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:55.899974 7487 raft_consensus.cc:504] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7e500b1b82446180190c6a2a15be3b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36171 } } peers { permanent_uuid: "2dcd1054f86c4b06b5e909da54bec822" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43189 } } peers { permanent_uuid: "aa07792917bb4ac6b56ff5606093e8a8" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35673 } }
I20260925 00:42:55.900384 7487 leader_election.cc:288] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [CANDIDATE]: Term 1 election: Requested vote from peers 3d7e500b1b82446180190c6a2a15be3b (127.6.72.67:36171), aa07792917bb4ac6b56ff5606093e8a8 (127.6.72.65:35673)
I20260925 00:42:55.900765 7416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c012ab364b246e28bcafcf58da17cf2" candidate_uuid: "2dcd1054f86c4b06b5e909da54bec822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d7e500b1b82446180190c6a2a15be3b"
I20260925 00:42:55.900856 7148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c012ab364b246e28bcafcf58da17cf2" candidate_uuid: "2dcd1054f86c4b06b5e909da54bec822" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa07792917bb4ac6b56ff5606093e8a8"
I20260925 00:42:55.900942 7416 raft_consensus.cc:3037] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:55.901048 7148 raft_consensus.cc:3037] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:55.902081 7416 raft_consensus.cc:2454] T 6c012ab364b246e28bcafcf58da17cf2 P 3d7e500b1b82446180190c6a2a15be3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dcd1054f86c4b06b5e909da54bec822 in term 1.
I20260925 00:42:55.902202 7148 raft_consensus.cc:2454] T 6c012ab364b246e28bcafcf58da17cf2 P aa07792917bb4ac6b56ff5606093e8a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dcd1054f86c4b06b5e909da54bec822 in term 1.
I20260925 00:42:55.902407 7217 leader_election.cc:302] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dcd1054f86c4b06b5e909da54bec822, 3d7e500b1b82446180190c6a2a15be3b; no voters:
I20260925 00:42:55.902683 7487 raft_consensus.cc:2781] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:42:55.902940 7487 raft_consensus.cc:686] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [term 1 LEADER]: Becoming Leader. State: Replica: 2dcd1054f86c4b06b5e909da54bec822, State: Running, Role: LEADER
I20260925 00:42:55.903328 7487 consensus_queue.cc:237] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7e500b1b82446180190c6a2a15be3b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36171 } } peers { permanent_uuid: "2dcd1054f86c4b06b5e909da54bec822" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43189 } } peers { permanent_uuid: "aa07792917bb4ac6b56ff5606093e8a8" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35673 } }
I20260925 00:42:55.906621 7003 catalog_manager.cc:5495] T 6c012ab364b246e28bcafcf58da17cf2 P 2dcd1054f86c4b06b5e909da54bec822 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2dcd1054f86c4b06b5e909da54bec822 (127.6.72.66). New cstate: current_term: 1 leader_uuid: "2dcd1054f86c4b06b5e909da54bec822" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7e500b1b82446180190c6a2a15be3b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dcd1054f86c4b06b5e909da54bec822" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa07792917bb4ac6b56ff5606093e8a8" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35673 } health_report { overall_health: UNKNOWN } } }
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:42:55.920424 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.911174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.6.72.65:35673)
0925 00:42:55.911358 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.911363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.911497 (+ 134us) server_negotiation.cc:407] Connection header received
0925 00:42:55.911671 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.911674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.911730 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.911812 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:55.912558 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.913078 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.914320 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.914543 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.914649 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.917166 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:55.917178 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:55.917194 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:55.917221 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:55.918793 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.919207 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.919212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.919213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.919312 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.919558 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.919560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.919561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.919807 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:55.920018 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.920126 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.920227 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296975, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296975, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:42:55.949028 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.939937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37192 (local address 127.6.72.65:35673)
0925 00:42:55.940101 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.940105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.940174 (+ 69us) server_negotiation.cc:407] Connection header received
0925 00:42:55.940388 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.940393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.940443 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.940536 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:55.941308 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.941893 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.942724 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.942961 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.943072 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.945746 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:55.945749 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:55.945751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:55.945778 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:55.947054 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.947652 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.947656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.947658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.947751 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.948041 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.948044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.948046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.948283 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:55.948431 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.948732 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.948859 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 00:42:55 dist-test-slave-w8nh krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:42:55.976518 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:55.967165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37194 (local address 127.6.72.65:35673)
0925 00:42:55.967344 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:55.967350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:55.967490 (+ 140us) server_negotiation.cc:407] Connection header received
0925 00:42:55.967708 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:55.967712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:55.967766 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:55.967859 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:55.968630 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.969260 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.970186 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:55.970407 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:55.970519 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:55.972956 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:55.972960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:55.972963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:55.972999 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:55.974511 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.975079 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.975083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.975085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.975183 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:55.975538 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:55.975542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:55.975543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:55.975743 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:55.975894 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:55.976143 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 00:42:55.976322 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260925 00:42:55.977959 7108 tablet_service.cc:2908] Rejecting scan request for tablet 6c012ab364b246e28bcafcf58da17cf2: Uninitialized: clean time has not yet been initialized
I20260925 00:42:55.980129 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 7065
I20260925 00:42:55.986184 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 7199
I20260925 00:42:55.991622 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 7333
I20260925 00:42:55.997319 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 6971
2026-09-25T00:42:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2337 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:42:56 dist-test-slave-w8nh krb5kdc[7503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:42:56 dist-test-slave-w8nh krb5kdc[7503](info): set up 2 sockets
Sep 25 00:42:56 dist-test-slave-w8nh krb5kdc[7503](info): commencing operation
krb5kdc: starting...
W20260925 00:42:57.052812 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:42:57 dist-test-slave-w8nh krb5kdc[7503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:42:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:42:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:42:57.187923 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:42615
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:42615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:57.301936 7516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:57.302255 7516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:57.302297 7516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:57.306371 7516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:42:57.306442 7516 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 00:42:57.306460 7516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:57.306479 7516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:57.306495 7516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:42:57.306510 7516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:42:57.311949 7516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:42615
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:42615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:57.313135 7516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:57.314180 7516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:57.323096 7524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:57.323148 7521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:57.323160 7522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:57.323338 7516 server_base.cc:1034] running on GCE node
I20260925 00:42:57.324007 7516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:57.325182 7516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:57.326409 7516 hybrid_clock.cc:648] HybridClock initialized: now 1790296977326384 us; error 32 us; skew 500 ppm
Sep 25 00:42:57 dist-test-slave-w8nh krb5kdc[7503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296977, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:57.329483 7516 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:42:57.330480 7516 webserver.cc:460] Webserver started at http://127.6.72.126:38915/ using document root <none> and password file <none>
I20260925 00:42:57.331084 7516 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:57.331177 7516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:57.331422 7516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:57.333598 7516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "c65b46624df543288e82f0a3d01ae9b9"
format_stamp: "Formatted at 2026-09-25 00:42:57 on dist-test-slave-w8nh"
server_key: "449f021ec19dbb738730f5a4e3d801d6"
server_key_iv: "7e4dd8dd94ae7c6d5536a54d09102dee"
server_key_version: "encryptionkey@0"
I20260925 00:42:57.334195 7516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "c65b46624df543288e82f0a3d01ae9b9"
format_stamp: "Formatted at 2026-09-25 00:42:57 on dist-test-slave-w8nh"
server_key: "449f021ec19dbb738730f5a4e3d801d6"
server_key_iv: "7e4dd8dd94ae7c6d5536a54d09102dee"
server_key_version: "encryptionkey@0"
I20260925 00:42:57.338061 7516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 00:42:57.340572 7531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:57.341856 7516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 00:42:57.342010 7516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "c65b46624df543288e82f0a3d01ae9b9"
format_stamp: "Formatted at 2026-09-25 00:42:57 on dist-test-slave-w8nh"
server_key: "449f021ec19dbb738730f5a4e3d801d6"
server_key_iv: "7e4dd8dd94ae7c6d5536a54d09102dee"
server_key_version: "encryptionkey@0"
I20260925 00:42:57.342130 7516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:57.358697 7516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:57.362180 7516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:57.362413 7516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:57.370653 7516 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:42615
I20260925 00:42:57.370666 7593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:42615 every 8 connection(s)
I20260925 00:42:57.371651 7516 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:42:57.374469 7594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:57.374836 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 7516
I20260925 00:42:57.374969 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:42:57.375242 6433 external_mini_cluster.cc:1428] Setting key 6eb52834ebb79159ad1adf8ec9f22bfc
I20260925 00:42:57.381783 7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9: Bootstrap starting.
I20260925 00:42:57.384615 7594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9: Neither blocks nor log segments found. Creating new log.
Sep 25 00:42:57 dist-test-slave-w8nh krb5kdc[7503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:57.385345 7594 log.cc:824] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9: Log is configured to *not* fsync() on all Append() calls
I20260925 00:42:57.387527 7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9: No bootstrap required, opened a new log
I20260925 00:42:57.390673 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:57.376678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45338 (local address 127.6.72.126:42615)
0925 00:42:57.377068 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:57.377083 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:57.377118 (+ 35us) server_negotiation.cc:407] Connection header received
0925 00:42:57.377867 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:57.377886 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:57.378228 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:57.378509 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:57.379448 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.380541 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.381713 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.382289 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.382411 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:57.385308 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:57.385313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:57.385327 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:57.385354 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:57.387869 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:57.388374 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:57.388379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:57.388384 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:57.388490 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:57.388788 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:57.388791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:57.388793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:57.389187 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:57.389383 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:57.389664 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 00:42:57.389948 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":131,"threads_started":1}
I20260925 00:42:57.390597 7594 raft_consensus.cc:348] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65b46624df543288e82f0a3d01ae9b9" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42615 } }
I20260925 00:42:57.390866 7594 raft_consensus.cc:374] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:57.390944 7594 raft_consensus.cc:729] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c65b46624df543288e82f0a3d01ae9b9, State: Initialized, Role: FOLLOWER
I20260925 00:42:57.391373 7594 consensus_queue.cc:260] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65b46624df543288e82f0a3d01ae9b9" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42615 } }
I20260925 00:42:57.391530 7594 raft_consensus.cc:388] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:42:57.391621 7594 raft_consensus.cc:482] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:42:57.391734 7594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:57.392781 7594 raft_consensus.cc:504] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65b46624df543288e82f0a3d01ae9b9" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42615 } }
I20260925 00:42:57.393143 7594 leader_election.cc:302] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c65b46624df543288e82f0a3d01ae9b9; no voters:
I20260925 00:42:57.393450 7594 leader_election.cc:288] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:42:57.393608 7599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:42:57.393827 7599 raft_consensus.cc:686] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [term 1 LEADER]: Becoming Leader. State: Replica: c65b46624df543288e82f0a3d01ae9b9, State: Running, Role: LEADER
I20260925 00:42:57.394124 7599 consensus_queue.cc:237] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65b46624df543288e82f0a3d01ae9b9" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42615 } }
I20260925 00:42:57.394727 7594 sys_catalog.cc:564] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:42:57.395848 7600 sys_catalog.cc:455] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c65b46624df543288e82f0a3d01ae9b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65b46624df543288e82f0a3d01ae9b9" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42615 } } }
I20260925 00:42:57.395982 7600 sys_catalog.cc:458] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [sys.catalog]: This master's current role is: LEADER
I20260925 00:42:57.396317 7601 sys_catalog.cc:455] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c65b46624df543288e82f0a3d01ae9b9. Latest consensus state: current_term: 1 leader_uuid: "c65b46624df543288e82f0a3d01ae9b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c65b46624df543288e82f0a3d01ae9b9" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42615 } } }
I20260925 00:42:57.396402 7601 sys_catalog.cc:458] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9 [sys.catalog]: This master's current role is: LEADER
I20260925 00:42:57.396484 7608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:42:57.399379 7608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:42:57.405555 7608 catalog_manager.cc:1347] Generated new cluster ID: 229a23f6626b4fb9b865e37d28135689
I20260925 00:42:57.405644 7608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:42:57.436332 7608 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:42:57.437712 7608 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:42:57.443388 7608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9: Generated new TSK 0
I20260925 00:42:57.444280 7608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:42:57.478989 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:42615
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:57.594530 7620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:57.594889 7620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:57.594955 7620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:57.599296 7620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:57.599370 7620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:57.599412 7620 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 00:42:57.599490 7620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:42:57.605070 7620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.72.126:42615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:57.606241 7620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:57.607306 7620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 00:42:57.617635 7620 server_base.cc:1034] running on GCE node
W20260925 00:42:57.617647 7626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:57.617619 7628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:57.617666 7625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:57.618260 7620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:57.618914 7620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:57.620112 7620 hybrid_clock.cc:648] HybridClock initialized: now 1790296977620094 us; error 29 us; skew 500 ppm
Sep 25 00:42:57 dist-test-slave-w8nh krb5kdc[7503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296977, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:57.623040 7620 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:42:57.623971 7620 webserver.cc:460] Webserver started at http://127.6.72.65:39675/ using document root <none> and password file <none>
I20260925 00:42:57.624598 7620 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:57.624660 7620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:57.624830 7620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:57.626892 7620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "ff1ac6310bec408fa598d13e6ad31c37"
format_stamp: "Formatted at 2026-09-25 00:42:57 on dist-test-slave-w8nh"
server_key: "aba494419c11b49bbd45ef7ff182bebc"
server_key_iv: "b496e21ec23dba35619a15d200b81b08"
server_key_version: "encryptionkey@0"
I20260925 00:42:57.627417 7620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "ff1ac6310bec408fa598d13e6ad31c37"
format_stamp: "Formatted at 2026-09-25 00:42:57 on dist-test-slave-w8nh"
server_key: "aba494419c11b49bbd45ef7ff182bebc"
server_key_iv: "b496e21ec23dba35619a15d200b81b08"
server_key_version: "encryptionkey@0"
I20260925 00:42:57.631255 7620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260925 00:42:57.633833 7635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:57.635121 7620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:42:57.635221 7620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "ff1ac6310bec408fa598d13e6ad31c37"
format_stamp: "Formatted at 2026-09-25 00:42:57 on dist-test-slave-w8nh"
server_key: "aba494419c11b49bbd45ef7ff182bebc"
server_key_iv: "b496e21ec23dba35619a15d200b81b08"
server_key_version: "encryptionkey@0"
I20260925 00:42:57.635344 7620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:57.657160 7620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:57.660569 7620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:57.660738 7620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:57.662134 7620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:42:57.662248 7620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:57.662329 7620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:42:57.662379 7620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 00:42:57 dist-test-slave-w8nh krb5kdc[7503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296977, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:57.673770 7620 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:44293
I20260925 00:42:57.673804 7750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:44293 every 8 connection(s)
I20260925 00:42:57.674803 7620 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:42:57.676357 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 7620
I20260925 00:42:57.676476 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:42:57.676760 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:57.663218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:49831 (local address 127.6.72.126:42615)
0925 00:42:57.663354 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:57.663357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:57.664541 (+ 1184us) server_negotiation.cc:407] Connection header received
0925 00:42:57.665675 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:57.665679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:57.665742 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:57.665849 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:57.667292 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.667892 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.669145 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.669472 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.669585 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:57.672748 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:57.672752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:57.672755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:57.672788 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:57.674461 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:57.675056 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:57.675058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:57.675060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:57.675139 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:57.675552 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:57.675556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:57.675557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:57.675773 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:57.675853 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:57.676477 (+ 624us) server_negotiation.cc:299] Negotiation successful
0925 00:42:57.676622 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 00:42:57.676790 6433 external_mini_cluster.cc:1428] Setting key 818ebe6bb63b9eb1976fc555dba89496
I20260925 00:42:57.677714 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:57.663806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42615 (local address 127.6.72.65:49831)
0925 00:42:57.664392 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:57.664436 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:57.665453 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:57.666030 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:57.666038 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:57.666466 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:57.667132 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:57.667151 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.668014 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.668018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:57.668998 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:57.669007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.669649 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.669654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:57.669826 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:57.670502 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:57.670517 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:57.672537 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:57.674581 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:57.674587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:57.674602 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:57.674960 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:57.675265 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:57.675269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:57.675271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:57.675440 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:57.675848 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:57.675853 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:57.676118 (+ 265us) client_negotiation.cc:769] Sending connection context
0925 00:42:57.676369 (+ 251us) client_negotiation.cc:240] Negotiation successful
0925 00:42:57.676720 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":141,"threads_started":1}
I20260925 00:42:57.679278 7751 heartbeater.cc:347] Connected to a master server at 127.6.72.126:42615
I20260925 00:42:57.679427 7751 heartbeater.cc:464] Registering TS with master...
I20260925 00:42:57.680015 7751 heartbeater.cc:511] Master 127.6.72.126:42615 requested a full tablet report, sending...
I20260925 00:42:57.681862 7547 ts_manager.cc:194] Registered new tserver with Master: ff1ac6310bec408fa598d13e6ad31c37 (127.6.72.65:44293)
I20260925 00:42:57.683020 7547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:49831
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:42:57.691510 7758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:57.681637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42615 (local address 127.6.72.65:45557)
0925 00:42:57.683753 (+ 2116us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:57.683770 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:57.683886 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:57.684155 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:57.684160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:57.684471 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:57.684670 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:57.684683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.685678 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.685682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:57.686670 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:57.686684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.687184 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.687190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:57.687307 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:57.687946 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:57.687958 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:57.688355 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:57.690021 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:57.690028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:57.690031 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:57.690259 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:57.690558 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:57.690560 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:57.690561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:57.690644 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:57.691084 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:57.691086 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:57.691161 (+ 75us) client_negotiation.cc:769] Sending connection context
0925 00:42:57.691243 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 00:42:57.691356 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2003,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260925 00:42:57.691700 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:57.681677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:45557 (local address 127.6.72.126:42615)
0925 00:42:57.681809 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:57.681813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:57.683854 (+ 2041us) server_negotiation.cc:407] Connection header received
0925 00:42:57.683979 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:57.683982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:57.684030 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:57.684115 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:57.684809 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.685535 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.686822 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.687033 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.687131 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:57.688492 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:57.688496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:57.688499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:57.688522 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:57.689881 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:57.690345 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:57.690349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:57.690350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:57.690455 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:57.690739 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:57.690745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:57.690749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:57.690989 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:57.691094 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:57.691327 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 00:42:57.691460 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:42:57.707934 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:42615
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:57.819180 7760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:57.819442 7760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:57.819487 7760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:57.823654 7760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:57.823716 7760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:57.823756 7760 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 00:42:57.823855 7760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:42:57.829437 7760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.72.126:42615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:57.830524 7760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:57.831563 7760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:57.841742 7768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:57.842049 7766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:57.842061 7765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:57.842350 7760 server_base.cc:1034] running on GCE node
I20260925 00:42:57.842801 7760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:57.843518 7760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:57.844754 7760 hybrid_clock.cc:648] HybridClock initialized: now 1790296977844715 us; error 49 us; skew 500 ppm
Sep 25 00:42:57 dist-test-slave-w8nh krb5kdc[7503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296977, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:57.847687 7760 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:42:57.848675 7760 webserver.cc:460] Webserver started at http://127.6.72.66:45727/ using document root <none> and password file <none>
I20260925 00:42:57.849298 7760 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:57.849396 7760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:57.849634 7760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:57.851735 7760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "b2719c4b08fc4826b2ac6fd2ce064854"
format_stamp: "Formatted at 2026-09-25 00:42:57 on dist-test-slave-w8nh"
server_key: "86f2829ec4ff5ead52390884540781e3"
server_key_iv: "6ab652d1a06de69bed919a28fe4116c1"
server_key_version: "encryptionkey@0"
I20260925 00:42:57.852393 7760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "b2719c4b08fc4826b2ac6fd2ce064854"
format_stamp: "Formatted at 2026-09-25 00:42:57 on dist-test-slave-w8nh"
server_key: "86f2829ec4ff5ead52390884540781e3"
server_key_iv: "6ab652d1a06de69bed919a28fe4116c1"
server_key_version: "encryptionkey@0"
I20260925 00:42:57.856345 7760 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:42:57.858914 7775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:57.860338 7760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:42:57.860473 7760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "b2719c4b08fc4826b2ac6fd2ce064854"
format_stamp: "Formatted at 2026-09-25 00:42:57 on dist-test-slave-w8nh"
server_key: "86f2829ec4ff5ead52390884540781e3"
server_key_iv: "6ab652d1a06de69bed919a28fe4116c1"
server_key_version: "encryptionkey@0"
I20260925 00:42:57.860596 7760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:57.877123 7760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:57.880897 7760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:57.881116 7760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:57.882664 7760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:42:57.882740 7760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:57.882815 7760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:42:57.882866 7760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 00:42:57 dist-test-slave-w8nh krb5kdc[7503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296977, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:57.893729 7760 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:46127
I20260925 00:42:57.893798 7890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:46127 every 8 connection(s)
I20260925 00:42:57.894677 7760 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:42:57.896656 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:57.883480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:55403 (local address 127.6.72.126:42615)
0925 00:42:57.883627 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:57.883632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:57.884573 (+ 941us) server_negotiation.cc:407] Connection header received
0925 00:42:57.885631 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:57.885635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:57.885710 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:57.885768 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:57.887321 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.888064 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.889315 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.889551 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.889671 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:57.892310 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:57.892315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:57.892320 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:57.892350 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:57.894049 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:57.894547 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:57.894552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:57.894553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:57.894649 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:57.895048 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:57.895054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:57.895056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:57.895330 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:57.895457 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:57.896411 (+ 954us) server_negotiation.cc:299] Negotiation successful
0925 00:42:57.896536 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 00:42:57.897104 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:57.883931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42615 (local address 127.6.72.66:55403)
0925 00:42:57.884410 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:57.884459 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:57.885376 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:57.886241 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:57.886248 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:57.886689 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:57.887170 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:57.887183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.888178 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.888183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:57.889194 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:57.889203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.889725 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.889729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:57.889969 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:57.890663 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:57.890679 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:57.892106 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:57.894167 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:57.894172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:57.894186 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:57.894442 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:57.894800 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:57.894803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:57.894805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:57.894940 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:57.895441 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:57.895447 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:57.895750 (+ 303us) client_negotiation.cc:769] Sending connection context
0925 00:42:57.895962 (+ 212us) client_negotiation.cc:240] Negotiation successful
0925 00:42:57.896312 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":144,"threads_started":1}
I20260925 00:42:57.898414 7891 heartbeater.cc:347] Connected to a master server at 127.6.72.126:42615
I20260925 00:42:57.898584 7891 heartbeater.cc:464] Registering TS with master...
I20260925 00:42:57.899159 7891 heartbeater.cc:511] Master 127.6.72.126:42615 requested a full tablet report, sending...
I20260925 00:42:57.900303 7548 ts_manager.cc:194] Registered new tserver with Master: b2719c4b08fc4826b2ac6fd2ce064854 (127.6.72.66:46127)
I20260925 00:42:57.901082 7548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:55403
I20260925 00:42:57.904947 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 7760
I20260925 00:42:57.905076 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:42:57.905385 6433 external_mini_cluster.cc:1428] Setting key acd8a8b4eed57487781322ae7e2dabc9
I20260925 00:42:57.908381 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:57.900492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:49057 (local address 127.6.72.126:42615)
0925 00:42:57.900652 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:57.900657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:57.900879 (+ 222us) server_negotiation.cc:407] Connection header received
0925 00:42:57.900953 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:57.900956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:57.901001 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:57.901090 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:57.901558 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.902290 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.903103 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.903349 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.903441 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:57.904601 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:57.904603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:57.904605 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:57.904626 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:57.906462 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:57.907004 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:57.907008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:57.907010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:57.907109 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:57.907427 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:57.907432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:57.907434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:57.907675 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:57.907789 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:57.908059 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 00:42:57.908175 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 00:42:57.908420 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:57.900411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42615 (local address 127.6.72.66:49057)
0925 00:42:57.900781 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:57.900793 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:57.900885 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:57.901118 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:57.901120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:57.901324 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:57.901450 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:57.901457 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.902420 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.902423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:57.902986 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:57.902993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:57.903452 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:57.903454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:57.903520 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:57.904052 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:57.904064 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:57.904466 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:57.906576 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:57.906579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:57.906581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:57.906886 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:57.907228 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:57.907231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:57.907233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:57.907330 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:57.907776 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:57.907779 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:57.907893 (+ 114us) client_negotiation.cc:769] Sending connection context
0925 00:42:57.908007 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 00:42:57.908214 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:42:57.934660 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:42615
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:58.049007 7900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:58.049299 7900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:58.049382 7900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:58.053571 7900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:58.053644 7900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:58.053685 7900 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 00:42:58.053754 7900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:42:58.059314 7900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.72.126:42615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:58.060536 7900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:58.061610 7900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:58.071791 7906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:58.071772 7908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:58.071766 7900 server_base.cc:1034] running on GCE node
W20260925 00:42:58.071758 7905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:58.072554 7900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:58.073251 7900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:58.074465 7900 hybrid_clock.cc:648] HybridClock initialized: now 1790296978074440 us; error 36 us; skew 500 ppm
Sep 25 00:42:58 dist-test-slave-w8nh krb5kdc[7503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296978, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:58.077368 7900 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:42:58.078302 7900 webserver.cc:460] Webserver started at http://127.6.72.67:34399/ using document root <none> and password file <none>
I20260925 00:42:58.078886 7900 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:58.078979 7900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:58.079282 7900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:58.081504 7900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "ae9c8a49158b4f6aa0a14fc0de154013"
format_stamp: "Formatted at 2026-09-25 00:42:58 on dist-test-slave-w8nh"
server_key: "3b0db0848f125d9b854366116f61fb56"
server_key_iv: "73da94f9319dae035d726a21225bb44f"
server_key_version: "encryptionkey@0"
I20260925 00:42:58.082073 7900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "ae9c8a49158b4f6aa0a14fc0de154013"
format_stamp: "Formatted at 2026-09-25 00:42:58 on dist-test-slave-w8nh"
server_key: "3b0db0848f125d9b854366116f61fb56"
server_key_iv: "73da94f9319dae035d726a21225bb44f"
server_key_version: "encryptionkey@0"
I20260925 00:42:58.085896 7900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260925 00:42:58.089403 7915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:58.090652 7900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 00:42:58.090792 7900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "ae9c8a49158b4f6aa0a14fc0de154013"
format_stamp: "Formatted at 2026-09-25 00:42:58 on dist-test-slave-w8nh"
server_key: "3b0db0848f125d9b854366116f61fb56"
server_key_iv: "73da94f9319dae035d726a21225bb44f"
server_key_version: "encryptionkey@0"
I20260925 00:42:58.090911 7900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:58.107849 7900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:58.111142 7900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:58.111371 7900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:58.112973 7900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:42:58.113050 7900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:58.113127 7900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:42:58.113171 7900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 00:42:58 dist-test-slave-w8nh krb5kdc[7503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296978, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:58.124318 7900 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:44801
I20260925 00:42:58.124400 8030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:44801 every 8 connection(s)
I20260925 00:42:58.125337 7900 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:42:58.127451 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.113802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:35023 (local address 127.6.72.126:42615)
0925 00:42:58.113980 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.113988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.114859 (+ 871us) server_negotiation.cc:407] Connection header received
0925 00:42:58.115878 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.115882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.115936 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.115996 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:58.117516 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.118411 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.119902 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.120290 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.120401 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.123514 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:58.123518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:58.123521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:58.123550 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:58.125276 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.125814 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.125821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.125826 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.125918 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.126355 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.126358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.126360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.126551 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:58.126671 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.127146 (+ 475us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.127279 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 00:42:58.128229 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.114226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42615 (local address 127.6.72.67:35023)
0925 00:42:58.114685 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.114737 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.115615 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.116471 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.116478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.116815 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:58.117358 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.117372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.118559 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.118563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.119772 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.119783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.120448 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.120451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.120632 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.121337 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:58.121356 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:58.123323 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:58.125406 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:58.125412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:58.125428 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:58.125706 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:58.126020 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:58.126024 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:58.126025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:58.126252 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:58.126659 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:58.126665 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:58.126916 (+ 251us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.127102 (+ 186us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.127424 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":140,"threads_started":1}
I20260925 00:42:58.129530 8031 heartbeater.cc:347] Connected to a master server at 127.6.72.126:42615
I20260925 00:42:58.129705 8031 heartbeater.cc:464] Registering TS with master...
I20260925 00:42:58.130205 8031 heartbeater.cc:511] Master 127.6.72.126:42615 requested a full tablet report, sending...
I20260925 00:42:58.131337 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 7900
I20260925 00:42:58.131398 7547 ts_manager.cc:194] Registered new tserver with Master: ae9c8a49158b4f6aa0a14fc0de154013 (127.6.72.67:44801)
I20260925 00:42:58.131481 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:42:58.131752 6433 external_mini_cluster.cc:1428] Setting key 11279aaea53877b1af694c3b454bd17c
I20260925 00:42:58.132110 7547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:35023
I20260925 00:42:58.134030 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:42:58.140585 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.132408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:35463 (local address 127.6.72.126:42615)
0925 00:42:58.132621 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.132624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.132708 (+ 84us) server_negotiation.cc:407] Connection header received
0925 00:42:58.132880 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.132883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.132929 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.133014 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:58.133532 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.134261 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.135545 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.135773 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.135858 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.137177 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:58.137181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:58.137184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:58.137211 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:58.138749 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.139267 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.139273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.139276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.139375 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.139643 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.139646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.139648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.139901 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:58.139985 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.140358 (+ 373us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.140460 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260925 00:42:58.140780 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.132312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42615 (local address 127.6.72.67:35463)
0925 00:42:58.132628 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.132643 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.132778 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.133029 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.133032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.133285 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:58.133406 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.133414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.134413 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.134417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.135388 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.135402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.135897 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.135903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.136040 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.136680 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:58.136692 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:58.137034 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:58.138904 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:58.138906 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:58.138908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:58.139156 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:58.139474 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:58.139476 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:58.139477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:58.139550 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:58.139998 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:58.140004 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:58.140123 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.140285 (+ 162us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.140477 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":124,"threads_started":1}
Sep 25 00:42:58 dist-test-slave-w8nh krb5kdc[7503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296978, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 00:42:58 dist-test-slave-w8nh krb5kdc[7503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296978, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:58.164019 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.154612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.6.72.126:42615)
0925 00:42:58.154760 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.154763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.154775 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:42:58.154819 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.154821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.154859 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.154924 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:58.155824 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.156431 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.157300 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.157564 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.157691 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.160397 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:58.160400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:58.160402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:58.160435 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:58.162125 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.162600 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.162603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.162604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.162721 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.163026 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.163030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.163031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.163283 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:58.163401 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.163685 (+ 284us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.163796 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:42:58.167649 7548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:42:58.170857 7548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:42:58.186549 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.180477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:44801 (local address 127.0.0.1:36756)
0925 00:42:58.181093 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.181151 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.181284 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.181953 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.181959 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.181983 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:58.182133 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.182144 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.183555 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.183559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.184887 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.184896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.186011 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.186015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.186124 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.186181 (+ 57us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.186268 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.186390 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":101,"threads_started":1}
I20260925 00:42:58.187431 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.180652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.6.72.67:44801)
0925 00:42:58.181425 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.181437 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.181459 (+ 22us) server_negotiation.cc:407] Connection header received
0925 00:42:58.181531 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.181535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.181776 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.181963 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:58.182278 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.183424 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.185030 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.185885 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.186938 (+ 1053us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.187107 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.187172 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.187279 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":124,"threads_started":1}
I20260925 00:42:58.187489 8053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.182422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.6.72.65:44293)
0925 00:42:58.182698 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.182705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.182850 (+ 145us) server_negotiation.cc:407] Connection header received
0925 00:42:58.182939 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.182943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.183174 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.183335 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:58.183765 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.184876 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.185978 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.186586 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.186739 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.186910 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.187240 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.187345 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
I20260925 00:42:58.187444 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.182186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:44293 (local address 127.0.0.1:37172)
0925 00:42:58.182740 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.182753 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.182845 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.183317 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.183320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.183335 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:58.183423 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.183429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.184995 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.185002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.185837 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.185846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.186981 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.186985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.187079 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.187099 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.187202 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.187284 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":97,"threads_started":1}
I20260925 00:42:58.188190 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.180605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:46127 (local address 127.0.0.1:45612)
0925 00:42:58.182156 (+ 1551us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.182172 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.182277 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.182942 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.182944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.182957 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:58.183068 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.183073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.184568 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.184572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.185733 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.185743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.187811 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.187815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.187909 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.187931 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.188000 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.188060 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1480,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260925 00:42:58.188630 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.181440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45612 (local address 127.6.72.66:46127)
0925 00:42:58.181756 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.181767 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.182402 (+ 635us) server_negotiation.cc:407] Connection header received
0925 00:42:58.182510 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.182538 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.182755 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.182944 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:58.183192 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.184447 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.185875 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.187480 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.187587 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.187750 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.188400 (+ 650us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.188499 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":96,"threads_started":1}
I20260925 00:42:58.190577 7965 tablet_service.cc:1467] Processing CreateTablet for tablet 0039b0289ef8488a83899630b8a9db02 (DEFAULT_TABLE table=test-table [id=5c0026a0589f4a95a35a2348e530e86c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:42:58.191227 7685 tablet_service.cc:1467] Processing CreateTablet for tablet 0039b0289ef8488a83899630b8a9db02 (DEFAULT_TABLE table=test-table [id=5c0026a0589f4a95a35a2348e530e86c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:42:58.191270 7825 tablet_service.cc:1467] Processing CreateTablet for tablet 0039b0289ef8488a83899630b8a9db02 (DEFAULT_TABLE table=test-table [id=5c0026a0589f4a95a35a2348e530e86c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:42:58.191774 7965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0039b0289ef8488a83899630b8a9db02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:58.192359 7825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0039b0289ef8488a83899630b8a9db02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:58.192384 7685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0039b0289ef8488a83899630b8a9db02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:58.199858 8054 tablet_bootstrap.cc:492] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: Bootstrap starting.
I20260925 00:42:58.200838 8056 tablet_bootstrap.cc:492] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: Bootstrap starting.
I20260925 00:42:58.201404 8055 tablet_bootstrap.cc:492] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: Bootstrap starting.
I20260925 00:42:58.202883 8056 tablet_bootstrap.cc:654] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:58.202873 8054 tablet_bootstrap.cc:654] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:58.203490 8055 tablet_bootstrap.cc:654] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:58.203696 8056 log.cc:824] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: Log is configured to *not* fsync() on all Append() calls
I20260925 00:42:58.203797 8054 log.cc:824] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: Log is configured to *not* fsync() on all Append() calls
I20260925 00:42:58.204368 8055 log.cc:824] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: Log is configured to *not* fsync() on all Append() calls
I20260925 00:42:58.205924 8056 tablet_bootstrap.cc:492] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: No bootstrap required, opened a new log
I20260925 00:42:58.205921 8054 tablet_bootstrap.cc:492] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: No bootstrap required, opened a new log
I20260925 00:42:58.206120 8054 ts_tablet_manager.cc:1397] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:42:58.206120 8056 ts_tablet_manager.cc:1397] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 00:42:58.206146 8055 tablet_bootstrap.cc:492] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: No bootstrap required, opened a new log
I20260925 00:42:58.206362 8055 ts_tablet_manager.cc:1397] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 00:42:58.209566 8055 raft_consensus.cc:348] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } }
I20260925 00:42:58.209843 8055 raft_consensus.cc:374] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:58.209903 8055 raft_consensus.cc:729] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae9c8a49158b4f6aa0a14fc0de154013, State: Initialized, Role: FOLLOWER
I20260925 00:42:58.210127 8056 raft_consensus.cc:348] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } }
I20260925 00:42:58.210389 8056 raft_consensus.cc:374] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:58.210449 8056 raft_consensus.cc:729] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2719c4b08fc4826b2ac6fd2ce064854, State: Initialized, Role: FOLLOWER
I20260925 00:42:58.210417 8055 consensus_queue.cc:260] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } }
I20260925 00:42:58.210911 8056 consensus_queue.cc:260] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } }
I20260925 00:42:58.211148 8031 heartbeater.cc:503] Master 127.6.72.126:42615 was elected leader, sending a full tablet report...
I20260925 00:42:58.211575 8055 ts_tablet_manager.cc:1428] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 00:42:58.211650 8054 raft_consensus.cc:348] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } }
I20260925 00:42:58.211922 8054 raft_consensus.cc:374] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:58.211987 8054 raft_consensus.cc:729] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1ac6310bec408fa598d13e6ad31c37, State: Initialized, Role: FOLLOWER
I20260925 00:42:58.212234 8056 ts_tablet_manager.cc:1428] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 00:42:58.212353 7891 heartbeater.cc:503] Master 127.6.72.126:42615 was elected leader, sending a full tablet report...
I20260925 00:42:58.212461 8054 consensus_queue.cc:260] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } }
I20260925 00:42:58.213083 7751 heartbeater.cc:503] Master 127.6.72.126:42615 was elected leader, sending a full tablet report...
I20260925 00:42:58.213361 8054 ts_tablet_manager.cc:1428] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260925 00:42:58.215197 8061 raft_consensus.cc:482] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:42:58.215411 8061 raft_consensus.cc:504] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } }
I20260925 00:42:58.217429 8061 leader_election.cc:288] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae9c8a49158b4f6aa0a14fc0de154013 (127.6.72.67:44801), ff1ac6310bec408fa598d13e6ad31c37 (127.6.72.65:44293)
I20260925 00:42:58.221449 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.217611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:44801 (local address 127.6.72.66:60257)
0925 00:42:58.217721 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.217740 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.217829 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.218384 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.218390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.218412 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:58.218549 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.218557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.219383 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.219389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.220345 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.220354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.221016 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.221020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.221137 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.221159 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.221263 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.221339 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 00:42:58.221527 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.218053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:60257 (local address 127.6.72.67:44801)
0925 00:42:58.218185 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.218188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.218200 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:42:58.218237 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.218239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.218273 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.218335 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:58.218686 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.219257 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.220458 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.220895 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.221015 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.221084 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.221303 (+ 219us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.221388 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 00:42:58.222280 8053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.218053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:51023 (local address 127.6.72.65:44293)
0925 00:42:58.218209 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.218219 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.218230 (+ 11us) server_negotiation.cc:407] Connection header received
0925 00:42:58.218268 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.218270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.218307 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.218410 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:58.218764 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.219305 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.220934 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.221483 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.221600 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.221667 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.222078 (+ 411us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.222190 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 00:42:58.222283 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.217686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:44293 (local address 127.6.72.66:51023)
0925 00:42:58.217962 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.217975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.218060 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.218511 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.218515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.218530 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:58.218662 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.218668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.219431 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.219438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.220777 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.220787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.221845 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.221848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.221928 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.221947 (+ 19us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.222055 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.222130 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":125,"threads_started":1}
I20260925 00:42:58.222297 7985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0039b0289ef8488a83899630b8a9db02" candidate_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" is_pre_election: true
I20260925 00:42:58.222633 7985 raft_consensus.cc:2454] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2719c4b08fc4826b2ac6fd2ce064854 in term 0.
I20260925 00:42:58.223012 7705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0039b0289ef8488a83899630b8a9db02" candidate_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1ac6310bec408fa598d13e6ad31c37" is_pre_election: true
I20260925 00:42:58.223241 7778 leader_election.cc:302] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae9c8a49158b4f6aa0a14fc0de154013, b2719c4b08fc4826b2ac6fd2ce064854; no voters:
I20260925 00:42:58.223325 7705 raft_consensus.cc:2454] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2719c4b08fc4826b2ac6fd2ce064854 in term 0.
I20260925 00:42:58.223577 8062 raft_consensus.cc:2781] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:42:58.223732 8062 raft_consensus.cc:482] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:42:58.223799 8062 raft_consensus.cc:3037] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:58.225152 8062 raft_consensus.cc:504] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } }
I20260925 00:42:58.225536 8062 leader_election.cc:288] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [CANDIDATE]: Term 1 election: Requested vote from peers ae9c8a49158b4f6aa0a14fc0de154013 (127.6.72.67:44801), ff1ac6310bec408fa598d13e6ad31c37 (127.6.72.65:44293)
I20260925 00:42:58.225919 7705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0039b0289ef8488a83899630b8a9db02" candidate_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1ac6310bec408fa598d13e6ad31c37"
I20260925 00:42:58.225922 7985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0039b0289ef8488a83899630b8a9db02" candidate_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae9c8a49158b4f6aa0a14fc0de154013"
I20260925 00:42:58.226074 7705 raft_consensus.cc:3037] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:58.226090 7985 raft_consensus.cc:3037] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:58.227193 7705 raft_consensus.cc:2454] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2719c4b08fc4826b2ac6fd2ce064854 in term 1.
I20260925 00:42:58.227605 7777 leader_election.cc:302] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2719c4b08fc4826b2ac6fd2ce064854, ff1ac6310bec408fa598d13e6ad31c37; no voters:
I20260925 00:42:58.227622 7985 raft_consensus.cc:2454] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2719c4b08fc4826b2ac6fd2ce064854 in term 1.
I20260925 00:42:58.227818 8062 raft_consensus.cc:2781] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:42:58.227948 8062 raft_consensus.cc:686] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 LEADER]: Becoming Leader. State: Replica: b2719c4b08fc4826b2ac6fd2ce064854, State: Running, Role: LEADER
I20260925 00:42:58.228353 8062 consensus_queue.cc:237] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } }
I20260925 00:42:58.231904 7548 catalog_manager.cc:5495] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2719c4b08fc4826b2ac6fd2ce064854 (127.6.72.66). New cstate: current_term: 1 leader_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } health_report { overall_health: UNKNOWN } } }
I20260925 00:42:58.255487 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.251595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45616 (local address 127.6.72.66:46127)
0925 00:42:58.251721 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.251725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.251794 (+ 69us) server_negotiation.cc:407] Connection header received
0925 00:42:58.252008 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.252011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.252068 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.252156 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:42:58.252505 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.253013 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.253998 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.254206 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.254318 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.254695 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:42:58.255128 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:42:58.255214 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.255312 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.255371 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 00:42:58.262095 7705 raft_consensus.cc:1261] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37 [term 1 FOLLOWER]: Refusing update from remote peer b2719c4b08fc4826b2ac6fd2ce064854: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:42:58.262133 7985 raft_consensus.cc:1261] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 FOLLOWER]: Refusing update from remote peer b2719c4b08fc4826b2ac6fd2ce064854: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:42:58.263192 8061 consensus_queue.cc:1035] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:42:58.263387 8062 consensus_queue.cc:1035] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:42:58.270576 8067 mvcc.cc:205] Tried to move back new op lower bound from 7333056422952206336 to 7333056422826225664. Current Snapshot: MvccSnapshot[applied={T|T < 7333056422952206336}]
I20260925 00:42:58.283538 7548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:42:58.283763 7548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:42:58.286481 7548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9: Sending DeleteTablet for 3 replicas of tablet 0039b0289ef8488a83899630b8a9db02
I20260925 00:42:58.287339 7965 tablet_service.cc:1514] Processing DeleteTablet for tablet 0039b0289ef8488a83899630b8a9db02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:42:58 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:36756
I20260925 00:42:58.287463 7685 tablet_service.cc:1514] Processing DeleteTablet for tablet 0039b0289ef8488a83899630b8a9db02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:42:58 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:37172
I20260925 00:42:58.288192 8077 tablet_replica.cc:330] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: stopping tablet replica
I20260925 00:42:58.288184 7825 tablet_service.cc:1514] Processing DeleteTablet for tablet 0039b0289ef8488a83899630b8a9db02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:42:58 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:45612
I20260925 00:42:58.288232 7548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 00:42:58.288573 8077 raft_consensus.cc:2229] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 00:42:58.288702 7548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:42:58.288914 8077 raft_consensus.cc:2258] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:42:58.289311 8076 tablet_replica.cc:330] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: stopping tablet replica
I20260925 00:42:58.289600 8076 raft_consensus.cc:2229] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:42:58.289911 8076 raft_consensus.cc:2258] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:42:58.290174 8078 tablet_replica.cc:330] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: stopping tablet replica
I20260925 00:42:58.290439 8078 raft_consensus.cc:2229] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:42:58.290822 8078 raft_consensus.cc:2258] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:42:58.291494 8076 ts_tablet_manager.cc:1905] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:42:58.292308 8078 ts_tablet_manager.cc:1905] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:42:58.294173 7685 tablet_service.cc:1467] Processing CreateTablet for tablet a7df7563be8541d8b775a839ed40d165 (DEFAULT_TABLE table=test-table [id=cd91431c7d9e46c1ae74bc7cf88f03e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:42:58.294443 8076 ts_tablet_manager.cc:1918] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:42:58.294497 7685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7df7563be8541d8b775a839ed40d165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:58.294560 8076 log.cc:1196] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/0039b0289ef8488a83899630b8a9db02
I20260925 00:42:58.294992 8076 ts_tablet_manager.cc:1939] T 0039b0289ef8488a83899630b8a9db02 P ae9c8a49158b4f6aa0a14fc0de154013: Deleting consensus metadata
I20260925 00:42:58.294984 8077 ts_tablet_manager.cc:1905] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:42:58.295089 8078 ts_tablet_manager.cc:1918] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:42:58.295181 8078 log.cc:1196] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/0039b0289ef8488a83899630b8a9db02
I20260925 00:42:58.295504 8078 ts_tablet_manager.cc:1939] T 0039b0289ef8488a83899630b8a9db02 P b2719c4b08fc4826b2ac6fd2ce064854: Deleting consensus metadata
I20260925 00:42:58.296680 7534 catalog_manager.cc:4841] TS ae9c8a49158b4f6aa0a14fc0de154013 (127.6.72.67:44801): tablet 0039b0289ef8488a83899630b8a9db02 (table test-table [id=5c0026a0589f4a95a35a2348e530e86c]) successfully deleted
I20260925 00:42:58.296788 7825 tablet_service.cc:1467] Processing CreateTablet for tablet a7df7563be8541d8b775a839ed40d165 (DEFAULT_TABLE table=test-table [id=cd91431c7d9e46c1ae74bc7cf88f03e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:42:58.297046 7825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7df7563be8541d8b775a839ed40d165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:58.297155 7533 catalog_manager.cc:4841] TS b2719c4b08fc4826b2ac6fd2ce064854 (127.6.72.66:46127): tablet 0039b0289ef8488a83899630b8a9db02 (table test-table [id=5c0026a0589f4a95a35a2348e530e86c]) successfully deleted
I20260925 00:42:58.297258 8054 tablet_bootstrap.cc:492] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37: Bootstrap starting.
I20260925 00:42:58.297269 7965 tablet_service.cc:1467] Processing CreateTablet for tablet a7df7563be8541d8b775a839ed40d165 (DEFAULT_TABLE table=test-table [id=cd91431c7d9e46c1ae74bc7cf88f03e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:42:58.297501 7965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7df7563be8541d8b775a839ed40d165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:58.298010 8054 tablet_bootstrap.cc:654] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:58.299356 8054 tablet_bootstrap.cc:492] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37: No bootstrap required, opened a new log
I20260925 00:42:58.299448 8054 ts_tablet_manager.cc:1397] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 00:42:58.299962 8055 tablet_bootstrap.cc:492] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: Bootstrap starting.
I20260925 00:42:58.299933 8054 raft_consensus.cc:348] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.300036 8054 raft_consensus.cc:374] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:58.300072 8054 raft_consensus.cc:729] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1ac6310bec408fa598d13e6ad31c37, State: Initialized, Role: FOLLOWER
I20260925 00:42:58.300215 8054 consensus_queue.cc:260] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.300493 8054 ts_tablet_manager.cc:1428] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:42:58.300992 8077 ts_tablet_manager.cc:1918] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:42:58.301119 8077 log.cc:1196] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/0039b0289ef8488a83899630b8a9db02
I20260925 00:42:58.301882 8077 ts_tablet_manager.cc:1939] T 0039b0289ef8488a83899630b8a9db02 P ff1ac6310bec408fa598d13e6ad31c37: Deleting consensus metadata
I20260925 00:42:58.301931 8055 tablet_bootstrap.cc:654] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:58.302628 8056 tablet_bootstrap.cc:492] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: Bootstrap starting.
I20260925 00:42:58.303151 8055 tablet_bootstrap.cc:492] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: No bootstrap required, opened a new log
I20260925 00:42:58.303237 8055 ts_tablet_manager.cc:1397] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 00:42:58.303211 7533 catalog_manager.cc:4841] TS ff1ac6310bec408fa598d13e6ad31c37 (127.6.72.65:44293): tablet 0039b0289ef8488a83899630b8a9db02 (table test-table [id=5c0026a0589f4a95a35a2348e530e86c]) successfully deleted
I20260925 00:42:58.303742 8056 tablet_bootstrap.cc:654] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:58.303683 8055 raft_consensus.cc:348] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.303833 8055 raft_consensus.cc:374] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:58.303874 8055 raft_consensus.cc:729] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae9c8a49158b4f6aa0a14fc0de154013, State: Initialized, Role: FOLLOWER
I20260925 00:42:58.304018 8055 consensus_queue.cc:260] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.305338 8055 ts_tablet_manager.cc:1428] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260925 00:42:58.305572 8056 tablet_bootstrap.cc:492] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: No bootstrap required, opened a new log
I20260925 00:42:58.305672 8056 ts_tablet_manager.cc:1397] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260925 00:42:58.306058 8056 raft_consensus.cc:348] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.306216 8056 raft_consensus.cc:374] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:58.306259 8056 raft_consensus.cc:729] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2719c4b08fc4826b2ac6fd2ce064854, State: Initialized, Role: FOLLOWER
I20260925 00:42:58.306407 8056 consensus_queue.cc:260] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.306643 8056 ts_tablet_manager.cc:1428] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 00:42:58.324661 8060 raft_consensus.cc:482] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:42:58.324867 8060 raft_consensus.cc:504] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.325999 8060 leader_election.cc:288] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff1ac6310bec408fa598d13e6ad31c37 (127.6.72.65:44293), b2719c4b08fc4826b2ac6fd2ce064854 (127.6.72.66:46127)
I20260925 00:42:58.330626 7924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.326268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:44293 (local address 127.6.72.67:52495)
0925 00:42:58.326503 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.326520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.326612 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.327012 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.327017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.327044 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:58.327164 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.327174 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.328039 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.328045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.329242 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.329253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.330170 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.330174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.330277 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.330304 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.330387 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.330471 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":51}
I20260925 00:42:58.330622 8053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.326357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:52495 (local address 127.6.72.65:44293)
0925 00:42:58.326533 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.326538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.326560 (+ 22us) server_negotiation.cc:407] Connection header received
0925 00:42:58.326768 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.326771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.326826 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.326896 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:58.327289 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.327931 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.329362 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.330025 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.330150 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.330219 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.330415 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.330490 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 00:42:58.331079 8082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.326531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:46127 (local address 127.6.72.67:52361)
0925 00:42:58.326998 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.327011 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.327085 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.327453 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.327455 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.327473 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:42:58.327578 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.327583 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.328600 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.328604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.329711 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.329719 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.330579 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.330582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.330677 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.330699 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.330812 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.330903 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260925 00:42:58.331223 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.326930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:52361 (local address 127.6.72.66:46127)
0925 00:42:58.327047 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.327051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.327065 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:42:58.327228 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.327234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.327283 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.327331 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:42:58.327705 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.328484 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.329823 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.330363 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.330454 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.330510 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.331028 (+ 518us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.331110 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:42:58.331347 7705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7df7563be8541d8b775a839ed40d165" candidate_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1ac6310bec408fa598d13e6ad31c37" is_pre_election: true
I20260925 00:42:58.331518 7705 raft_consensus.cc:2454] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae9c8a49158b4f6aa0a14fc0de154013 in term 0.
I20260925 00:42:58.331725 7845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7df7563be8541d8b775a839ed40d165" candidate_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" is_pre_election: true
I20260925 00:42:58.331898 7917 leader_election.cc:302] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae9c8a49158b4f6aa0a14fc0de154013, ff1ac6310bec408fa598d13e6ad31c37; no voters:
I20260925 00:42:58.331975 7845 raft_consensus.cc:2454] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae9c8a49158b4f6aa0a14fc0de154013 in term 0.
I20260925 00:42:58.332183 8060 raft_consensus.cc:2781] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:42:58.332283 8060 raft_consensus.cc:482] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:42:58.332329 8060 raft_consensus.cc:3037] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:58.333288 8060 raft_consensus.cc:504] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.333578 8060 leader_election.cc:288] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [CANDIDATE]: Term 1 election: Requested vote from peers ff1ac6310bec408fa598d13e6ad31c37 (127.6.72.65:44293), b2719c4b08fc4826b2ac6fd2ce064854 (127.6.72.66:46127)
I20260925 00:42:58.333988 7705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7df7563be8541d8b775a839ed40d165" candidate_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1ac6310bec408fa598d13e6ad31c37"
I20260925 00:42:58.334118 7705 raft_consensus.cc:3037] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:58.334206 7845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7df7563be8541d8b775a839ed40d165" candidate_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2719c4b08fc4826b2ac6fd2ce064854"
I20260925 00:42:58.334352 7845 raft_consensus.cc:3037] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:58.335297 7705 raft_consensus.cc:2454] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae9c8a49158b4f6aa0a14fc0de154013 in term 1.
I20260925 00:42:58.335412 7845 raft_consensus.cc:2454] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae9c8a49158b4f6aa0a14fc0de154013 in term 1.
I20260925 00:42:58.335602 7917 leader_election.cc:302] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae9c8a49158b4f6aa0a14fc0de154013, ff1ac6310bec408fa598d13e6ad31c37; no voters:
I20260925 00:42:58.335867 8060 raft_consensus.cc:2781] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:42:58.336066 8060 raft_consensus.cc:686] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 LEADER]: Becoming Leader. State: Replica: ae9c8a49158b4f6aa0a14fc0de154013, State: Running, Role: LEADER
I20260925 00:42:58.336375 8060 consensus_queue.cc:237] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.338963 7547 catalog_manager.cc:5495] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae9c8a49158b4f6aa0a14fc0de154013 (127.6.72.67). New cstate: current_term: 1 leader_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } health_report { overall_health: HEALTHY } } }
Sep 25 00:42:58 dist-test-slave-w8nh krb5kdc[7503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296977, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
W20260925 00:42:58.376432 8032 tablet.cc:2369] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:42:58.379053 8090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.370168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42615 (local address 127.0.0.1:59660)
0925 00:42:58.370586 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.370599 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.370744 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.370991 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.370994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.371416 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:42:58.371512 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.371520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.372282 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.372285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.372979 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.372990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.373432 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.373435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.373514 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.374126 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:42:58.374138 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:42:58.375555 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:42:58.377375 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:58.377386 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:58.377392 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:58.377629 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:58.377916 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:42:58.377921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:42:58.377923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:42:58.378003 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:42:58.378404 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:42:58.378412 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:42:58.378524 (+ 112us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.378638 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.378857 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":1,"thread_start_us":121,"threads_started":1}
I20260925 00:42:58.379053 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.370269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.6.72.126:42615)
0925 00:42:58.370435 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.370440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.370682 (+ 242us) server_negotiation.cc:407] Connection header received
0925 00:42:58.370842 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.370845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.370889 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.370974 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:58.371633 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.372160 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.373103 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.373321 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.373429 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.375715 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:58.375718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:58.375720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:58.375740 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:58.377243 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.377739 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.377744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.377746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.377822 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.378091 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.378096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.378098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.378289 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:58.378394 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.378694 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.378907 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":71}
I20260925 00:42:58.383670 7547 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:59660:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 00:42:58.384538 7547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:42:58.389685 7685 tablet_service.cc:1467] Processing CreateTablet for tablet 8f178df75f4e41bea3d4683fd32dc2ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d5779f748a44d67b21129a0e0ccca33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 00:42:58.389685 7825 tablet_service.cc:1467] Processing CreateTablet for tablet 8f178df75f4e41bea3d4683fd32dc2ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d5779f748a44d67b21129a0e0ccca33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 00:42:58.389804 7965 tablet_service.cc:1467] Processing CreateTablet for tablet 8f178df75f4e41bea3d4683fd32dc2ae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d5779f748a44d67b21129a0e0ccca33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 00:42:58.390095 7825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f178df75f4e41bea3d4683fd32dc2ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:58.390095 7685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f178df75f4e41bea3d4683fd32dc2ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:58.390172 7965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f178df75f4e41bea3d4683fd32dc2ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:58.393198 8054 tablet_bootstrap.cc:492] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: Bootstrap starting.
I20260925 00:42:58.393544 8055 tablet_bootstrap.cc:492] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: Bootstrap starting.
I20260925 00:42:58.394274 8054 tablet_bootstrap.cc:654] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:58.394490 8055 tablet_bootstrap.cc:654] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:58.395284 8056 tablet_bootstrap.cc:492] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: Bootstrap starting.
I20260925 00:42:58.396211 8056 tablet_bootstrap.cc:654] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:58.396386 8054 tablet_bootstrap.cc:492] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: No bootstrap required, opened a new log
I20260925 00:42:58.396430 8055 tablet_bootstrap.cc:492] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: No bootstrap required, opened a new log
I20260925 00:42:58.396459 8054 ts_tablet_manager.cc:1397] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 00:42:58.396505 8055 ts_tablet_manager.cc:1397] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 00:42:58.396893 8054 raft_consensus.cc:348] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.396997 8054 raft_consensus.cc:374] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:58.397027 8054 raft_consensus.cc:729] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1ac6310bec408fa598d13e6ad31c37, State: Initialized, Role: FOLLOWER
I20260925 00:42:58.397145 8054 consensus_queue.cc:260] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
W20260925 00:42:58.397303 7892 tablet.cc:2369] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:42:58.397434 8056 tablet_bootstrap.cc:492] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: No bootstrap required, opened a new log
I20260925 00:42:58.397354 8055 raft_consensus.cc:348] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.397477 8055 raft_consensus.cc:374] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:58.397423 8063 tablet_replica.cc:439] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } }
I20260925 00:42:58.397511 8056 ts_tablet_manager.cc:1397] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260925 00:42:58.397512 8055 raft_consensus.cc:729] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae9c8a49158b4f6aa0a14fc0de154013, State: Initialized, Role: FOLLOWER
I20260925 00:42:58.397543 8063 tablet_replica.cc:442] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:42:58.397651 8055 consensus_queue.cc:260] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.397938 8055 ts_tablet_manager.cc:1428] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:42:58.397986 8056 raft_consensus.cc:348] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.398087 8056 raft_consensus.cc:374] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:42:58.398123 8056 raft_consensus.cc:729] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2719c4b08fc4826b2ac6fd2ce064854, State: Initialized, Role: FOLLOWER
I20260925 00:42:58.398100 8060 tablet_replica.cc:439] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } }
I20260925 00:42:58.398259 8060 tablet_replica.cc:442] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:42:58.398272 8056 consensus_queue.cc:260] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.398505 8056 ts_tablet_manager.cc:1428] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:42:58.398701 8061 tablet_replica.cc:439] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } }
I20260925 00:42:58.399298 8061 tablet_replica.cc:442] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:42:58.400938 8054 ts_tablet_manager.cc:1428] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
W20260925 00:42:58.426275 7752 tablet.cc:2369] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:42:58.453593 8060 raft_consensus.cc:482] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:42:58.453758 8060 raft_consensus.cc:504] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.454175 8060 leader_election.cc:288] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff1ac6310bec408fa598d13e6ad31c37 (127.6.72.65:44293), b2719c4b08fc4826b2ac6fd2ce064854 (127.6.72.66:46127)
I20260925 00:42:58.454753 7705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f178df75f4e41bea3d4683fd32dc2ae" candidate_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1ac6310bec408fa598d13e6ad31c37" is_pre_election: true
I20260925 00:42:58.454752 7845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f178df75f4e41bea3d4683fd32dc2ae" candidate_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" is_pre_election: true
I20260925 00:42:58.454910 7705 raft_consensus.cc:2454] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae9c8a49158b4f6aa0a14fc0de154013 in term 0.
I20260925 00:42:58.454921 7845 raft_consensus.cc:2454] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae9c8a49158b4f6aa0a14fc0de154013 in term 0.
I20260925 00:42:58.455229 7917 leader_election.cc:302] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae9c8a49158b4f6aa0a14fc0de154013, ff1ac6310bec408fa598d13e6ad31c37; no voters:
I20260925 00:42:58.455427 8060 raft_consensus.cc:2781] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:42:58.455490 8060 raft_consensus.cc:482] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:42:58.455531 8060 raft_consensus.cc:3037] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:58.456573 8060 raft_consensus.cc:504] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.456934 8060 leader_election.cc:288] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [CANDIDATE]: Term 1 election: Requested vote from peers ff1ac6310bec408fa598d13e6ad31c37 (127.6.72.65:44293), b2719c4b08fc4826b2ac6fd2ce064854 (127.6.72.66:46127)
I20260925 00:42:58.457350 7845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f178df75f4e41bea3d4683fd32dc2ae" candidate_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2719c4b08fc4826b2ac6fd2ce064854"
I20260925 00:42:58.457453 7845 raft_consensus.cc:3037] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:58.457716 7705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f178df75f4e41bea3d4683fd32dc2ae" candidate_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1ac6310bec408fa598d13e6ad31c37"
I20260925 00:42:58.457839 7705 raft_consensus.cc:3037] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:42:58.458617 7845 raft_consensus.cc:2454] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae9c8a49158b4f6aa0a14fc0de154013 in term 1.
I20260925 00:42:58.458930 7917 leader_election.cc:302] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae9c8a49158b4f6aa0a14fc0de154013, b2719c4b08fc4826b2ac6fd2ce064854; no voters:
I20260925 00:42:58.458977 7705 raft_consensus.cc:2454] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae9c8a49158b4f6aa0a14fc0de154013 in term 1.
I20260925 00:42:58.459100 8060 raft_consensus.cc:2781] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:42:58.459179 8060 raft_consensus.cc:686] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 LEADER]: Becoming Leader. State: Replica: ae9c8a49158b4f6aa0a14fc0de154013, State: Running, Role: LEADER
I20260925 00:42:58.459365 8060 consensus_queue.cc:237] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } }
I20260925 00:42:58.460114 8083 tablet_replica.cc:439] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: TxnStatusTablet state changed. Reason: New leader ae9c8a49158b4f6aa0a14fc0de154013. Latest consensus state: current_term: 1 leader_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } }
I20260925 00:42:58.460219 8083 tablet_replica.cc:442] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: This TxnStatusTablet replica's current role is: LEADER
I20260925 00:42:58.460989 8095 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 00:42:58.461179 8095 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 00:42:58.461552 7547 catalog_manager.cc:5495] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae9c8a49158b4f6aa0a14fc0de154013 (127.6.72.67). New cstate: current_term: 1 leader_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } health_report { overall_health: HEALTHY } } }
I20260925 00:42:58.486188 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.481969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.6.72.67:44801)
0925 00:42:58.482082 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.482086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.482101 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:42:58.482215 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.482218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.482281 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.482388 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:42:58.482676 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.483530 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.484524 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.484866 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.484977 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.485267 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:42:58.485804 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:42:58.485917 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.485988 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.486044 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 00:42:58.486158 8090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.481827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:44801 (local address 127.0.0.1:42008)
0925 00:42:58.481975 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.481993 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.482092 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.482436 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.482439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.482447 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:42:58.482553 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.482558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.483655 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.483658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.484402 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.484412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.485000 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.485003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.485076 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.485122 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:42:58.485902 (+ 780us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:42:58.485916 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.485974 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.486043 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 00:42:58.489248 7705 raft_consensus.cc:1261] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37 [term 1 FOLLOWER]: Refusing update from remote peer ae9c8a49158b4f6aa0a14fc0de154013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:42:58.489475 7845 raft_consensus.cc:1261] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 FOLLOWER]: Refusing update from remote peer ae9c8a49158b4f6aa0a14fc0de154013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:42:58.490084 8083 consensus_queue.cc:1035] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:42:58.490273 8060 consensus_queue.cc:1035] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:42:58.491816 8063 tablet_replica.cc:439] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: TxnStatusTablet state changed. Reason: New leader ae9c8a49158b4f6aa0a14fc0de154013. Latest consensus state: current_term: 1 leader_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } }
I20260925 00:42:58.491940 8063 tablet_replica.cc:442] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:42:58.491915 8061 tablet_replica.cc:439] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: TxnStatusTablet state changed. Reason: New leader ae9c8a49158b4f6aa0a14fc0de154013. Latest consensus state: current_term: 1 leader_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } }
I20260925 00:42:58.492031 8061 tablet_replica.cc:442] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:42:58.494256 8068 mvcc.cc:205] Tried to move back new op lower bound from 7333056423886667776 to 7333056423771021312. Current Snapshot: MvccSnapshot[applied={T|T < 7333056423886667776}]
I20260925 00:42:58.495820 8061 tablet_replica.cc:439] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } }
I20260925 00:42:58.495949 8061 tablet_replica.cc:442] T 8f178df75f4e41bea3d4683fd32dc2ae P b2719c4b08fc4826b2ac6fd2ce064854: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:42:58.495919 8083 tablet_replica.cc:439] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } }
I20260925 00:42:58.496073 8083 tablet_replica.cc:442] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: This TxnStatusTablet replica's current role is: LEADER
I20260925 00:42:58.497295 8060 tablet_replica.cc:439] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } }
I20260925 00:42:58.497447 8060 tablet_replica.cc:442] T 8f178df75f4e41bea3d4683fd32dc2ae P ae9c8a49158b4f6aa0a14fc0de154013: This TxnStatusTablet replica's current role is: LEADER
I20260925 00:42:58.498466 8063 tablet_replica.cc:439] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 } } peers { permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 } } peers { permanent_uuid: "ae9c8a49158b4f6aa0a14fc0de154013" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44801 } } }
I20260925 00:42:58.498598 8063 tablet_replica.cc:442] T 8f178df75f4e41bea3d4683fd32dc2ae P ff1ac6310bec408fa598d13e6ad31c37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:42:58.505460 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.501942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.6.72.67:44801)
0925 00:42:58.502067 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.502072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.502090 (+ 18us) server_negotiation.cc:407] Connection header received
0925 00:42:58.502299 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.502302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.502351 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.502433 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:42:58.502639 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.503416 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.504280 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.504618 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.504719 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.504950 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:42:58.505028 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:42:58.505099 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.505269 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.505319 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 00:42:58.516426 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.512810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:44801 (local address 127.6.72.67:35069)
0925 00:42:58.512972 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 00:42:58.512985 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:42:58.513063 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:42:58.513294 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:42:58.513296 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:42:58.513302 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:42:58.513398 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.513402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.514436 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.514445 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.515088 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:42:58.515097 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.515678 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.515681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:42:58.515765 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.515808 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:42:58.516095 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:42:58.516099 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 00:42:58.516147 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 00:42:58.516285 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":19}
I20260925 00:42:58.516430 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.512852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:35069 (local address 127.6.72.67:44801)
0925 00:42:58.512956 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.512960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.513030 (+ 70us) server_negotiation.cc:407] Connection header received
0925 00:42:58.513153 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.513156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.513209 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.513278 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:42:58.513499 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.514306 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.515226 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.515557 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.515655 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.515937 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:42:58.516017 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:42:58.516086 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.516163 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.516258 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":42}
I20260925 00:42:58.523187 7705 raft_consensus.cc:1261] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37 [term 1 FOLLOWER]: Refusing update from remote peer ae9c8a49158b4f6aa0a14fc0de154013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:42:58.523180 7845 raft_consensus.cc:1261] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 FOLLOWER]: Refusing update from remote peer ae9c8a49158b4f6aa0a14fc0de154013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:42:58.523610 8100 consensus_queue.cc:1035] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1ac6310bec408fa598d13e6ad31c37" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:42:58.523823 8060 consensus_queue.cc:1035] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2719c4b08fc4826b2ac6fd2ce064854" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:42:58.563292 7546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:42:58.563472 7546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:42:58.565798 7546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c65b46624df543288e82f0a3d01ae9b9: Sending DeleteTablet for 3 replicas of tablet a7df7563be8541d8b775a839ed40d165
I20260925 00:42:58.566599 7964 tablet_service.cc:1514] Processing DeleteTablet for tablet a7df7563be8541d8b775a839ed40d165 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:42:58 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:36756
I20260925 00:42:58.566591 7685 tablet_service.cc:1514] Processing DeleteTablet for tablet a7df7563be8541d8b775a839ed40d165 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:42:58 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:37172
I20260925 00:42:58.566808 8077 tablet_replica.cc:330] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37: stopping tablet replica
I20260925 00:42:58.566941 8077 raft_consensus.cc:2229] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:42:58.566965 7825 tablet_service.cc:1514] Processing DeleteTablet for tablet a7df7563be8541d8b775a839ed40d165 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:42:58 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:45612
I20260925 00:42:58.567087 8077 raft_consensus.cc:2258] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:42:58.567134 8078 tablet_replica.cc:330] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: stopping tablet replica
I20260925 00:42:58.567251 8078 raft_consensus.cc:2229] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:42:58.567272 8076 tablet_replica.cc:330] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: stopping tablet replica
I20260925 00:42:58.567373 8078 raft_consensus.cc:2258] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:42:58.567378 8076 raft_consensus.cc:2229] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:42:58.567703 8076 raft_consensus.cc:2258] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:42:58.568399 8077 ts_tablet_manager.cc:1905] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:42:58.568450 8076 ts_tablet_manager.cc:1905] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:42:58.568466 8078 ts_tablet_manager.cc:1905] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:42:58.571355 8078 ts_tablet_manager.cc:1918] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:42:58.571352 8076 ts_tablet_manager.cc:1918] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:42:58.571352 8077 ts_tablet_manager.cc:1918] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:42:58.571436 8078 log.cc:1196] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/a7df7563be8541d8b775a839ed40d165
I20260925 00:42:58.571436 8076 log.cc:1196] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/a7df7563be8541d8b775a839ed40d165
I20260925 00:42:58.571436 8077 log.cc:1196] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/a7df7563be8541d8b775a839ed40d165
I20260925 00:42:58.571709 8076 ts_tablet_manager.cc:1939] T a7df7563be8541d8b775a839ed40d165 P ae9c8a49158b4f6aa0a14fc0de154013: Deleting consensus metadata
I20260925 00:42:58.571722 8077 ts_tablet_manager.cc:1939] T a7df7563be8541d8b775a839ed40d165 P ff1ac6310bec408fa598d13e6ad31c37: Deleting consensus metadata
I20260925 00:42:58.571729 8078 ts_tablet_manager.cc:1939] T a7df7563be8541d8b775a839ed40d165 P b2719c4b08fc4826b2ac6fd2ce064854: Deleting consensus metadata
I20260925 00:42:58.572588 7533 catalog_manager.cc:4841] TS b2719c4b08fc4826b2ac6fd2ce064854 (127.6.72.66:46127): tablet a7df7563be8541d8b775a839ed40d165 (table test-table [id=cd91431c7d9e46c1ae74bc7cf88f03e2]) successfully deleted
I20260925 00:42:58.572790 7534 catalog_manager.cc:4841] TS ae9c8a49158b4f6aa0a14fc0de154013 (127.6.72.67:44801): tablet a7df7563be8541d8b775a839ed40d165 (table test-table [id=cd91431c7d9e46c1ae74bc7cf88f03e2]) successfully deleted
I20260925 00:42:58.573266 7533 catalog_manager.cc:4841] TS ff1ac6310bec408fa598d13e6ad31c37 (127.6.72.65:44293): tablet a7df7563be8541d8b775a839ed40d165 (table test-table [id=cd91431c7d9e46c1ae74bc7cf88f03e2]) successfully deleted
Sep 25 00:42:58 dist-test-slave-w8nh krb5kdc[7503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296978, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:42:58.586263 8053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.574432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.6.72.65:44293)
0925 00:42:58.574563 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.574567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.574748 (+ 181us) server_negotiation.cc:407] Connection header received
0925 00:42:58.574827 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.574830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.574883 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.574979 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:58.575622 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.576438 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.577256 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.577592 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.577709 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.581815 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:58.581835 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:58.581852 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:58.581895 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:58.584086 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.584667 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.584676 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.584678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.584793 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.585140 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.585143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.585144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.585491 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:58.585716 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.585940 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.586073 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 00:42:58.586760 7745 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34518
I20260925 00:42:58.597540 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:58.588785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59666 (local address 127.6.72.126:42615)
0925 00:42:58.588919 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:58.588925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:58.589000 (+ 75us) server_negotiation.cc:407] Connection header received
0925 00:42:58.589110 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:58.589114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:58.589186 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:58.589273 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:58.590157 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.590946 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.591780 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:58.592121 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:58.592266 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:58.593744 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:58.593748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:58.593751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:58.593783 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:58.595622 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.596176 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.596186 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.596188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.596335 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:58.596604 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:58.596607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:58.596609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:58.596861 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:58.597000 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:58.597230 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 00:42:58.597359 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260925 00:42:58.598014 7546 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59666
I20260925 00:42:58.599632 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 7620
I20260925 00:42:58.607993 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 7760
I20260925 00:42:58.615348 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 7900
I20260925 00:42:58.622191 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 7516
2026-09-25T00:42:58Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2624 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:42:58 dist-test-slave-w8nh krb5kdc[8112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:42:58 dist-test-slave-w8nh krb5kdc[8112](info): set up 2 sockets
Sep 25 00:42:58 dist-test-slave-w8nh krb5kdc[8112](info): commencing operation
krb5kdc: starting...
W20260925 00:42:59.668846 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:42:59 dist-test-slave-w8nh krb5kdc[8112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:42:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:42:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:42:59.795972 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41533
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:41533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:42:59.907652 8125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:42:59.907948 8125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:42:59.907992 8125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:42:59.912153 8125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:42:59.912227 8125 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 00:42:59.912263 8125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:42:59.912331 8125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:42:59.912364 8125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:42:59.912400 8125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:42:59.917928 8125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:41533
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:42:59.919173 8125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:42:59.920295 8125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:42:59.929039 8133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:59.929049 8130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:42:59.929049 8131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:42:59.929323 8125 server_base.cc:1034] running on GCE node
I20260925 00:42:59.929950 8125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:42:59.931124 8125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:42:59.932335 8125 hybrid_clock.cc:648] HybridClock initialized: now 1790296979932233 us; error 126 us; skew 500 ppm
Sep 25 00:42:59 dist-test-slave-w8nh krb5kdc[8112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296979, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:42:59.935232 8125 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:42:59.936127 8125 webserver.cc:460] Webserver started at http://127.6.72.126:40435/ using document root <none> and password file <none>
I20260925 00:42:59.936728 8125 fs_manager.cc:359] Metadata directory not provided
I20260925 00:42:59.936789 8125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:42:59.936955 8125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:42:59.939146 8125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "443b32f18e8e43de8944f891092fc2a1"
format_stamp: "Formatted at 2026-09-25 00:42:59 on dist-test-slave-w8nh"
server_key: "12cc3e4c61185c06e7fa0db8cf7fb5af"
server_key_iv: "234458b68e95c91344b75e0be4b85976"
server_key_version: "encryptionkey@0"
I20260925 00:42:59.939844 8125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "443b32f18e8e43de8944f891092fc2a1"
format_stamp: "Formatted at 2026-09-25 00:42:59 on dist-test-slave-w8nh"
server_key: "12cc3e4c61185c06e7fa0db8cf7fb5af"
server_key_iv: "234458b68e95c91344b75e0be4b85976"
server_key_version: "encryptionkey@0"
I20260925 00:42:59.944569 8125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260925 00:42:59.947039 8140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:42:59.948441 8125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:42:59.948576 8125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "443b32f18e8e43de8944f891092fc2a1"
format_stamp: "Formatted at 2026-09-25 00:42:59 on dist-test-slave-w8nh"
server_key: "12cc3e4c61185c06e7fa0db8cf7fb5af"
server_key_iv: "234458b68e95c91344b75e0be4b85976"
server_key_version: "encryptionkey@0"
I20260925 00:42:59.948704 8125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:42:59.967489 8125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:42:59.971303 8125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:42:59.971521 8125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:42:59.980028 8125 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:41533
I20260925 00:42:59.980062 8202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:41533 every 8 connection(s)
I20260925 00:42:59.981101 8125 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:42:59.982940 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 8125
I20260925 00:42:59.983078 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:42:59.983381 6433 external_mini_cluster.cc:1428] Setting key 38e614664b32762ccdd02792e5559f85
I20260925 00:42:59.984817 8203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:42:59.990778 8203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1: Bootstrap starting.
Sep 25 00:42:59 dist-test-slave-w8nh krb5kdc[8112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:42:59.993683 8203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1: Neither blocks nor log segments found. Creating new log.
I20260925 00:42:59.994848 8203 log.cc:824] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1: Log is configured to *not* fsync() on all Append() calls
I20260925 00:42:59.997377 8203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1: No bootstrap required, opened a new log
I20260925 00:42:59.998522 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:42:59.984839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34232 (local address 127.6.72.126:41533)
0925 00:42:59.985322 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0925 00:42:59.985344 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 00:42:59.985580 (+ 236us) server_negotiation.cc:407] Connection header received
0925 00:42:59.986362 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:42:59.986380 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:42:59.986718 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:42:59.987070 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:42:59.987903 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:59.988984 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:59.989889 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:42:59.990379 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:42:59.990551 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:42:59.993453 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:42:59.993459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:42:59.993475 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:42:59.993505 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:42:59.995487 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:59.996083 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:59.996088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:59.996093 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:59.996206 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:42:59.996586 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:42:59.996588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:42:59.996589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:42:59.996992 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:42:59.997142 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 00:42:59.997500 (+ 358us) server_negotiation.cc:299] Negotiation successful
0925 00:42:59.997768 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":160,"threads_started":1}
I20260925 00:43:00.001559 8203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443b32f18e8e43de8944f891092fc2a1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41533 } }
I20260925 00:43:00.001811 8203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:00.001890 8203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443b32f18e8e43de8944f891092fc2a1, State: Initialized, Role: FOLLOWER
I20260925 00:43:00.002354 8203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443b32f18e8e43de8944f891092fc2a1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41533 } }
I20260925 00:43:00.002506 8203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:00.002569 8203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:00.002696 8203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:00.003695 8203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443b32f18e8e43de8944f891092fc2a1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41533 } }
I20260925 00:43:00.004050 8203 leader_election.cc:302] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 443b32f18e8e43de8944f891092fc2a1; no voters:
I20260925 00:43:00.004456 8203 leader_election.cc:288] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:00.004582 8208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:00.004846 8208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [term 1 LEADER]: Becoming Leader. State: Replica: 443b32f18e8e43de8944f891092fc2a1, State: Running, Role: LEADER
I20260925 00:43:00.005162 8208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443b32f18e8e43de8944f891092fc2a1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41533 } }
I20260925 00:43:00.005774 8203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:00.006917 8210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 443b32f18e8e43de8944f891092fc2a1. Latest consensus state: current_term: 1 leader_uuid: "443b32f18e8e43de8944f891092fc2a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443b32f18e8e43de8944f891092fc2a1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41533 } } }
I20260925 00:43:00.007043 8210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:00.007329 8209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "443b32f18e8e43de8944f891092fc2a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443b32f18e8e43de8944f891092fc2a1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41533 } } }
I20260925 00:43:00.007416 8209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:00.007545 8217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:00.011054 8217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:00.017199 8217 catalog_manager.cc:1347] Generated new cluster ID: 49da07c75fc84a0da72dbd65dbcab358
I20260925 00:43:00.017280 8217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:00.037338 8217 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:00.038408 8217 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:00.045452 8217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1: Generated new TSK 0
I20260925 00:43:00.046267 8217 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:43:00.079866 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41533
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:00.196924 8229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:00.197223 8229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:00.197276 8229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:00.201349 8229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:00.201411 8229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:00.201452 8229 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 00:43:00.201467 8229 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 00:43:00.201499 8229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:00.207026 8229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.72.126:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:00.208189 8229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:00.209358 8229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:00.219141 8234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:00.219300 8229 server_base.cc:1034] running on GCE node
W20260925 00:43:00.219264 8235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:00.219564 8237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:00.220042 8229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:00.220759 8229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:00.221961 8229 hybrid_clock.cc:648] HybridClock initialized: now 1790296980221941 us; error 34 us; skew 500 ppm
Sep 25 00:43:00 dist-test-slave-w8nh krb5kdc[8112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296980, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:00.225118 8229 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:43:00.225973 8229 webserver.cc:460] Webserver started at http://127.6.72.65:42891/ using document root <none> and password file <none>
I20260925 00:43:00.226497 8229 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:00.226554 8229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:00.226723 8229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:00.228922 8229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "0d7c45c8384e4b07938ffa7a97f33a0f"
format_stamp: "Formatted at 2026-09-25 00:43:00 on dist-test-slave-w8nh"
server_key: "3dd6093ee11540a6f1c9b08b1f70fa92"
server_key_iv: "6a13c3b4b2ceb64dd88fbf2475f1306b"
server_key_version: "encryptionkey@0"
I20260925 00:43:00.229424 8229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "0d7c45c8384e4b07938ffa7a97f33a0f"
format_stamp: "Formatted at 2026-09-25 00:43:00 on dist-test-slave-w8nh"
server_key: "3dd6093ee11540a6f1c9b08b1f70fa92"
server_key_iv: "6a13c3b4b2ceb64dd88fbf2475f1306b"
server_key_version: "encryptionkey@0"
I20260925 00:43:00.233067 8229 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:43:00.235531 8244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:00.237031 8229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:00.237149 8229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "0d7c45c8384e4b07938ffa7a97f33a0f"
format_stamp: "Formatted at 2026-09-25 00:43:00 on dist-test-slave-w8nh"
server_key: "3dd6093ee11540a6f1c9b08b1f70fa92"
server_key_iv: "6a13c3b4b2ceb64dd88fbf2475f1306b"
server_key_version: "encryptionkey@0"
I20260925 00:43:00.237254 8229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:00.274734 8229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:00.277802 8229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:00.277964 8229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:00.279385 8229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:00.279482 8229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:00.279556 8229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:00.279609 8229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 00:43:00 dist-test-slave-w8nh krb5kdc[8112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296980, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:00.290977 8229 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:38829
I20260925 00:43:00.291046 8359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:38829 every 8 connection(s)
I20260925 00:43:00.291921 8229 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:00.295341 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.280486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:34885 (local address 127.6.72.126:41533)
0925 00:43:00.280640 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.280643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.281809 (+ 1166us) server_negotiation.cc:407] Connection header received
0925 00:43:00.283028 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.283033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.283108 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.283216 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:00.284860 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.285760 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.287619 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.287955 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.288077 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.291230 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:00.291234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:00.291236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:00.291265 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:00.293096 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.293745 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.293754 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.293755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.293850 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.294193 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.294196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.294197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.294388 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:00.294471 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.295024 (+ 553us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.295198 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:43:00.296219 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.281102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41533 (local address 127.6.72.65:34885)
0925 00:43:00.281633 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.281679 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.282767 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.283422 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.283433 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.283958 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:00.284683 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.284706 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.286000 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.286011 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.287472 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.287482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.288119 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.288123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.288347 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.289044 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:00.289068 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:00.290968 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:00.293269 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.293274 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.293286 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.293565 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.293945 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.293947 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.293949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.294087 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.294483 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:00.294488 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:00.294743 (+ 255us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.294901 (+ 158us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.295169 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":143,"threads_started":1}
I20260925 00:43:00.297271 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 8229
I20260925 00:43:00.297399 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:00.297672 6433 external_mini_cluster.cc:1428] Setting key 17fc2314cb3f6a8cdbe39aa1355ad0b8
I20260925 00:43:00.297936 8360 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41533
I20260925 00:43:00.298092 8360 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:00.298743 8360 heartbeater.cc:511] Master 127.6.72.126:41533 requested a full tablet report, sending...
I20260925 00:43:00.300489 8156 ts_manager.cc:194] Registered new tserver with Master: 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65:38829)
I20260925 00:43:00.301872 8156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:34885
I20260925 00:43:00.308467 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.300281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41533 (local address 127.6.72.65:58047)
0925 00:43:00.300604 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.300616 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.300739 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.301161 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.301165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.301444 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:00.301602 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.301611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.302680 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.302683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.303286 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.303295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.303768 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.303772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.303872 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.304462 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:00.304481 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:00.304849 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:00.306836 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.306839 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.306842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.307057 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.307424 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.307427 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.307429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.307516 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.307888 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:00.307891 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:00.307988 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.308092 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.308243 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":88,"threads_started":1}
I20260925 00:43:00.308463 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.300318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:58047 (local address 127.6.72.126:41533)
0925 00:43:00.300446 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.300451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.300702 (+ 251us) server_negotiation.cc:407] Connection header received
0925 00:43:00.300825 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.300828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.300919 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.301012 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:00.301747 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.302471 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.303408 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.303649 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.303764 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.305016 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:00.305019 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:00.305022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:00.305047 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:00.306724 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.307164 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.307167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.307169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.307275 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.307605 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.307611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.307612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.307792 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:00.307902 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.308142 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.308288 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:43:00.328014 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41533
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:00.445695 8369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:00.446028 8369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:00.446108 8369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:00.452046 8369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:00.452322 8369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:00.452404 8369 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 00:43:00.452442 8369 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 00:43:00.452515 8369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:00.463660 8369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.72.126:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:00.465374 8369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:00.466662 8369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:00.479743 8377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:00.479827 8374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:00.480103 8369 server_base.cc:1034] running on GCE node
W20260925 00:43:00.479743 8375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:00.480881 8369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:00.481771 8369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:00.482977 8369 hybrid_clock.cc:648] HybridClock initialized: now 1790296980482949 us; error 41 us; skew 500 ppm
Sep 25 00:43:00 dist-test-slave-w8nh krb5kdc[8112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296980, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:00.486874 8369 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:43:00.488005 8369 webserver.cc:460] Webserver started at http://127.6.72.66:42419/ using document root <none> and password file <none>
I20260925 00:43:00.488698 8369 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:00.488790 8369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:00.489022 8369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:00.491776 8369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "ca961619ac6547939cb7a50a8b68db59"
format_stamp: "Formatted at 2026-09-25 00:43:00 on dist-test-slave-w8nh"
server_key: "2cf02a7ad0b4d485a91bc0eda6cca6ec"
server_key_iv: "b93e75adf8d4f8110b6aa8bbbf8469b0"
server_key_version: "encryptionkey@0"
I20260925 00:43:00.492447 8369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "ca961619ac6547939cb7a50a8b68db59"
format_stamp: "Formatted at 2026-09-25 00:43:00 on dist-test-slave-w8nh"
server_key: "2cf02a7ad0b4d485a91bc0eda6cca6ec"
server_key_iv: "b93e75adf8d4f8110b6aa8bbbf8469b0"
server_key_version: "encryptionkey@0"
I20260925 00:43:00.497433 8369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 00:43:00.500033 8384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:00.501338 8369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:00.501474 8369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "ca961619ac6547939cb7a50a8b68db59"
format_stamp: "Formatted at 2026-09-25 00:43:00 on dist-test-slave-w8nh"
server_key: "2cf02a7ad0b4d485a91bc0eda6cca6ec"
server_key_iv: "b93e75adf8d4f8110b6aa8bbbf8469b0"
server_key_version: "encryptionkey@0"
I20260925 00:43:00.501582 8369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:00.522920 8369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:00.529923 8369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:00.530154 8369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:00.531735 8369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:00.531814 8369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:00.531896 8369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:00.531945 8369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 00:43:00 dist-test-slave-w8nh krb5kdc[8112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296980, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:00.544366 8369 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:32935
I20260925 00:43:00.544442 8499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:32935 every 8 connection(s)
I20260925 00:43:00.545467 8369 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:00.547370 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.532599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:47535 (local address 127.6.72.126:41533)
0925 00:43:00.532727 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.532730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.533980 (+ 1250us) server_negotiation.cc:407] Connection header received
0925 00:43:00.535062 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.535066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.535136 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.535206 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:00.537165 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.537776 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.539115 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.539427 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.539556 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.542924 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:00.542928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:00.542931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:00.542972 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:00.544916 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.545510 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.545515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.545517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.545627 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.546013 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.546017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.546019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.546293 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:00.546401 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.547058 (+ 657us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.547238 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 00:43:00.548174 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.533230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41533 (local address 127.6.72.66:47535)
0925 00:43:00.533803 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.533857 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.534794 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.535710 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.535719 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.536291 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:00.537002 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.537028 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.537901 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.537905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.538966 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.538976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.539614 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.539619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.539806 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.540554 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:00.540586 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:00.542731 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:00.545048 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.545074 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.545091 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.545390 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.545738 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.545744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.545746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.545897 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.546460 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:00.546466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:00.546802 (+ 336us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.546970 (+ 168us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.547256 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":154,"threads_started":1}
I20260925 00:43:00.549633 8500 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41533
I20260925 00:43:00.549825 8500 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:00.550449 8500 heartbeater.cc:511] Master 127.6.72.126:41533 requested a full tablet report, sending...
I20260925 00:43:00.551555 8156 ts_manager.cc:194] Registered new tserver with Master: ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935)
I20260925 00:43:00.552358 8156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:47535
I20260925 00:43:00.555717 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 8369
I20260925 00:43:00.555850 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:00.556156 6433 external_mini_cluster.cc:1428] Setting key 06da0050fa9efeaf8331eac78ce68cc6
I20260925 00:43:00.560388 8506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.551744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41533 (local address 127.6.72.66:56231)
0925 00:43:00.552017 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.552030 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.552104 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.552987 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.552989 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.553219 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:00.553324 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.553339 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.554376 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.554379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.555022 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.555030 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.555520 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.555526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.555643 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.556198 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:00.556214 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:00.556675 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:00.558738 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.558741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.558743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.558963 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.559277 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.559287 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.559288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.559363 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.559893 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:00.559895 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:00.559970 (+ 75us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.560050 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.560159 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":94,"threads_started":1}
I20260925 00:43:00.560480 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.552386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:56231 (local address 127.6.72.126:41533)
0925 00:43:00.552517 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.552522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.552534 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:43:00.552565 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.552567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.552601 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.552697 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:00.553469 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.554200 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.555133 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.555347 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.555426 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.556838 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:00.556841 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:00.556844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:00.556867 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:00.558625 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.559063 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.559069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.559074 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.559176 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.559457 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.559462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.559466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.559798 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:00.559905 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.560133 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.560298 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:43:00.584136 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41533
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:00.696525 8509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:00.696857 8509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:00.696987 8509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:00.701335 8509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:00.701431 8509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:00.701512 8509 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 00:43:00.701557 8509 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 00:43:00.701622 8509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:00.707152 8509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.72.126:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:00.708279 8509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:00.709337 8509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:00.719449 8517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:00.719449 8514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:00.719509 8515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:00.719785 8509 server_base.cc:1034] running on GCE node
I20260925 00:43:00.720188 8509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:00.720952 8509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:00.722136 8509 hybrid_clock.cc:648] HybridClock initialized: now 1790296980722090 us; error 59 us; skew 500 ppm
Sep 25 00:43:00 dist-test-slave-w8nh krb5kdc[8112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296980, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:00.725307 8509 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:43:00.726234 8509 webserver.cc:460] Webserver started at http://127.6.72.67:38329/ using document root <none> and password file <none>
I20260925 00:43:00.726881 8509 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:00.726974 8509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:00.727221 8509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:00.729360 8509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac"
format_stamp: "Formatted at 2026-09-25 00:43:00 on dist-test-slave-w8nh"
server_key: "024660028e60a4cb0e214aeeea8f4b48"
server_key_iv: "0cb1042e0f5d45d5f42b10560787c63a"
server_key_version: "encryptionkey@0"
I20260925 00:43:00.730041 8509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac"
format_stamp: "Formatted at 2026-09-25 00:43:00 on dist-test-slave-w8nh"
server_key: "024660028e60a4cb0e214aeeea8f4b48"
server_key_iv: "0cb1042e0f5d45d5f42b10560787c63a"
server_key_version: "encryptionkey@0"
I20260925 00:43:00.733925 8509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:43:00.736402 8524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:00.737604 8509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 00:43:00.737759 8509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac"
format_stamp: "Formatted at 2026-09-25 00:43:00 on dist-test-slave-w8nh"
server_key: "024660028e60a4cb0e214aeeea8f4b48"
server_key_iv: "0cb1042e0f5d45d5f42b10560787c63a"
server_key_version: "encryptionkey@0"
I20260925 00:43:00.737875 8509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:00.752988 8509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:00.756356 8509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:00.756557 8509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:00.758009 8509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:00.758081 8509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:00.758169 8509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:00.758219 8509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 00:43:00 dist-test-slave-w8nh krb5kdc[8112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296980, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:00.768986 8509 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:45787
I20260925 00:43:00.769110 8639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:45787 every 8 connection(s)
I20260925 00:43:00.770112 8509 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:00.770629 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 8509
I20260925 00:43:00.770757 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:00.771123 6433 external_mini_cluster.cc:1428] Setting key 286c4a28a44a8ee1240b60c4c0a56162
I20260925 00:43:00.772904 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.758774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:38665 (local address 127.6.72.126:41533)
0925 00:43:00.758926 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.758930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.759854 (+ 924us) server_negotiation.cc:407] Connection header received
0925 00:43:00.761056 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.761060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.761127 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.761183 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:00.762736 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.763479 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.765558 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.765862 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.765952 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.769029 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:00.769034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:00.769038 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:00.769071 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:00.770561 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.771179 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.771186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.771189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.771289 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.771658 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.771662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.771664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.771915 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:00.772035 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.772604 (+ 569us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.772725 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 00:43:00.773743 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.759236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41533 (local address 127.6.72.67:38665)
0925 00:43:00.759686 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.759735 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.760768 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.761658 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.761664 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.762043 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:00.762588 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.762601 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.763627 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.763633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.765408 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.765422 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.766021 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.766025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.766262 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.766991 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:00.767018 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:00.768842 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:00.770702 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.770708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.770724 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.771066 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.771386 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.771389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.771391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.771547 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.772030 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:00.772036 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:00.772362 (+ 326us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.772548 (+ 186us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.772855 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":129,"threads_started":1}
I20260925 00:43:00.774797 8640 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41533
I20260925 00:43:00.774928 8640 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:00.775458 8640 heartbeater.cc:511] Master 127.6.72.126:41533 requested a full tablet report, sending...
I20260925 00:43:00.776569 8157 ts_manager.cc:194] Registered new tserver with Master: eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787)
I20260925 00:43:00.777397 8157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:38665
I20260925 00:43:00.783725 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.777049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41533 (local address 127.6.72.67:40841)
0925 00:43:00.777342 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.777356 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.777453 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.777735 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.777738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.777961 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:00.778112 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.778125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.779094 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.779096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.779676 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.779684 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.780132 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.780135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.780209 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.780722 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:00.780733 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:00.781066 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:00.782508 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.782510 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.782512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.782692 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.782921 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:00.782923 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:00.782925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:00.782992 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:00.783286 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:00.783288 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:00.783360 (+ 72us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.783436 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.783537 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":17,"thread_start_us":93,"threads_started":1}
I20260925 00:43:00.783794 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.777130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:40841 (local address 127.6.72.126:41533)
0925 00:43:00.777252 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.777255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.777408 (+ 153us) server_negotiation.cc:407] Connection header received
0925 00:43:00.777548 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.777553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.777599 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.777701 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:00.778230 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.778948 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.779785 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.779987 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.780068 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.781187 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:00.781189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:00.781192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:00.781213 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:00.782406 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.782774 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.782776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.782777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.782847 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.783065 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.783067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.783068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.783212 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:00.783305 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.783518 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.783625 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:43:00.785173 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 00:43:00 dist-test-slave-w8nh krb5kdc[8112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 00:43:00 dist-test-slave-w8nh krb5kdc[8112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:00.811349 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.802436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34248 (local address 127.6.72.126:41533)
0925 00:43:00.802565 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.802569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.802582 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:43:00.802626 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.802629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.802669 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.802740 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:00.803558 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.804072 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.805055 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.805270 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.805359 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.808272 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:00.808294 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:00.808297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:00.808325 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:00.809751 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.810124 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.810127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.810128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.810203 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:00.810487 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:00.810490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:00.810491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:00.810665 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:00.810767 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.811012 (+ 245us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.811108 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 00:43:00.814368 8157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:00.817374 8157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:00.832072 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.826125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44226 (local address 127.6.72.65:38829)
0925 00:43:00.826449 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.826457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.826977 (+ 520us) server_negotiation.cc:407] Connection header received
0925 00:43:00.827187 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.827192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.827390 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.827509 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:00.828027 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.829132 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.830708 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.831440 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.831554 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.831709 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.831823 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.831925 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":95,"threads_started":1}
I20260925 00:43:00.832067 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.825742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:38829 (local address 127.0.0.1:44226)
0925 00:43:00.826887 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.826903 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.827033 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.827680 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.827682 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.827720 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:00.827912 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.827919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.829270 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.829273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.830580 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.830589 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.831563 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.831566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.831658 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.831688 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.831772 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.831899 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"mutex_wait_us":217,"spinlock_wait_cycles":384,"thread_start_us":91,"threads_started":1}
I20260925 00:43:00.832466 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.825696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:32935 (local address 127.0.0.1:32966)
0925 00:43:00.826822 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.826871 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.827005 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.827549 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.827555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.827577 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:00.827721 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.827730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.829270 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.829273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.830830 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.830840 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.831868 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.831871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.831946 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.831968 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.832027 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.832090 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":173,"thread_start_us":73,"threads_started":1}
I20260925 00:43:00.832813 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.825831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.6.72.66:32935)
0925 00:43:00.826675 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.826686 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.827139 (+ 453us) server_negotiation.cc:407] Connection header received
0925 00:43:00.827207 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.827211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.827390 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.827522 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:00.827853 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.829131 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.830958 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.831755 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.831857 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.832009 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.832056 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.832667 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":108,"threads_started":1}
I20260925 00:43:00.835306 8434 tablet_service.cc:1467] Processing CreateTablet for tablet b50a71d7786b4355b446842911e5cf5e (DEFAULT_TABLE table=test-table [id=58b916a9bc9c4e69948794d945129dca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:00.835304 8294 tablet_service.cc:1467] Processing CreateTablet for tablet b50a71d7786b4355b446842911e5cf5e (DEFAULT_TABLE table=test-table [id=58b916a9bc9c4e69948794d945129dca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:00.836478 8294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50a71d7786b4355b446842911e5cf5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:00.836478 8434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50a71d7786b4355b446842911e5cf5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:00.836877 8657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.826412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:45787 (local address 127.0.0.1:56824)
0925 00:43:00.827051 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.827064 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.827153 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.827681 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.827684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.827733 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:00.827917 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.827922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.829268 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.829272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.830829 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.830840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.836454 (+ 5614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.836458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.836564 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.836586 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.836654 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.836738 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":126,"spinlock_wait_cycles":58240,"thread_start_us":78,"threads_started":1}
I20260925 00:43:00.837677 8662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.826666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.6.72.67:45787)
0925 00:43:00.827182 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.827198 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.827222 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:43:00.827296 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.827300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.827437 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.827595 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:00.828384 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.829135 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.833027 (+ 3892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.836313 (+ 3286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.836453 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.836708 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.837312 (+ 604us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.837412 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":126,"threads_started":1}
I20260925 00:43:00.839879 8574 tablet_service.cc:1467] Processing CreateTablet for tablet b50a71d7786b4355b446842911e5cf5e (DEFAULT_TABLE table=test-table [id=58b916a9bc9c4e69948794d945129dca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:00.840935 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50a71d7786b4355b446842911e5cf5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:00.844128 8663 tablet_bootstrap.cc:492] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: Bootstrap starting.
I20260925 00:43:00.845441 8664 tablet_bootstrap.cc:492] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: Bootstrap starting.
I20260925 00:43:00.846640 8663 tablet_bootstrap.cc:654] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:00.847218 8664 tablet_bootstrap.cc:654] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:00.847718 8663 log.cc:824] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:00.848042 8664 log.cc:824] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:00.849064 8665 tablet_bootstrap.cc:492] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: Bootstrap starting.
I20260925 00:43:00.849756 8663 tablet_bootstrap.cc:492] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: No bootstrap required, opened a new log
I20260925 00:43:00.849960 8663 ts_tablet_manager.cc:1397] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:43:00.850801 8665 tablet_bootstrap.cc:654] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:00.851385 8665 log.cc:824] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:00.853229 8665 tablet_bootstrap.cc:492] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: No bootstrap required, opened a new log
I20260925 00:43:00.853415 8665 ts_tablet_manager.cc:1397] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 00:43:00.853869 8663 raft_consensus.cc:348] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:00.854103 8663 raft_consensus.cc:374] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:00.854168 8663 raft_consensus.cc:729] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d7c45c8384e4b07938ffa7a97f33a0f, State: Initialized, Role: FOLLOWER
I20260925 00:43:00.854562 8663 consensus_queue.cc:260] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:00.855238 8360 heartbeater.cc:503] Master 127.6.72.126:41533 was elected leader, sending a full tablet report...
I20260925 00:43:00.855567 8663 ts_tablet_manager.cc:1428] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:43:00.856974 8664 tablet_bootstrap.cc:492] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: No bootstrap required, opened a new log
I20260925 00:43:00.856894 8665 raft_consensus.cc:348] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:00.857064 8665 raft_consensus.cc:374] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:00.857102 8665 raft_consensus.cc:729] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf3cf8f75fb41ac8192cc5904cca3ac, State: Initialized, Role: FOLLOWER
I20260925 00:43:00.857173 8664 ts_tablet_manager.cc:1397] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.001s
I20260925 00:43:00.857465 8665 consensus_queue.cc:260] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:00.858001 8640 heartbeater.cc:503] Master 127.6.72.126:41533 was elected leader, sending a full tablet report...
I20260925 00:43:00.859189 8665 ts_tablet_manager.cc:1428] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:43:00.860956 8664 raft_consensus.cc:348] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:00.861267 8664 raft_consensus.cc:374] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:00.861325 8664 raft_consensus.cc:729] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca961619ac6547939cb7a50a8b68db59, State: Initialized, Role: FOLLOWER
I20260925 00:43:00.861726 8664 consensus_queue.cc:260] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:00.862346 8500 heartbeater.cc:503] Master 127.6.72.126:41533 was elected leader, sending a full tablet report...
I20260925 00:43:00.862629 8664 ts_tablet_manager.cc:1428] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260925 00:43:00.951737 8669 raft_consensus.cc:482] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:00.952021 8669 raft_consensus.cc:504] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:00.953511 8669 leader_election.cc:288] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935), eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787)
I20260925 00:43:00.957839 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.953841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:49915 (local address 127.6.72.66:32935)
0925 00:43:00.954007 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.954013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.954308 (+ 295us) server_negotiation.cc:407] Connection header received
0925 00:43:00.954517 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.954522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.954587 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.954683 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:00.955069 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.955632 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.956859 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.957280 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.957391 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.957459 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.957632 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.957710 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 00:43:00.957854 8662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.954392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:46701 (local address 127.6.72.67:45787)
0925 00:43:00.954521 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:00.954525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:00.954587 (+ 62us) server_negotiation.cc:407] Connection header received
0925 00:43:00.954786 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:00.954789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:00.954844 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:00.954966 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:00.955242 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.955763 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.956910 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.957289 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.957411 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.957473 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:00.957671 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 00:43:00.957752 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 00:43:00.957862 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.953759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:32935 (local address 127.6.72.65:49915)
0925 00:43:00.954121 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.954192 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.954338 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.954767 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.954771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.954794 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:00.954955 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.954964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.955757 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.955760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.956741 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.956751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.957406 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.957410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.957506 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.957532 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.957592 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.957679 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":21,"thread_start_us":167,"threads_started":1}
I20260925 00:43:00.957864 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:00.954218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:45787 (local address 127.6.72.65:46701)
0925 00:43:00.954493 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:00.954506 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:00.954603 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:00.954971 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:00.954973 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:00.954987 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:00.955119 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.955128 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.955872 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.955874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.956765 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:00.956773 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:00.957429 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:00.957432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:00.957534 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:00.957551 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 00:43:00.957645 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 00:43:00.957709 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":20,"thread_start_us":110,"threads_started":1}
I20260925 00:43:00.958645 8454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50a71d7786b4355b446842911e5cf5e" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca961619ac6547939cb7a50a8b68db59" is_pre_election: true
I20260925 00:43:00.958642 8594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50a71d7786b4355b446842911e5cf5e" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" is_pre_election: true
I20260925 00:43:00.958936 8594 raft_consensus.cc:2454] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 0.
I20260925 00:43:00.958945 8454 raft_consensus.cc:2454] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 0.
I20260925 00:43:00.959419 8246 leader_election.cc:302] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d7c45c8384e4b07938ffa7a97f33a0f, ca961619ac6547939cb7a50a8b68db59; no voters:
I20260925 00:43:00.959692 8669 raft_consensus.cc:2781] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:00.959779 8669 raft_consensus.cc:482] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:00.959821 8669 raft_consensus.cc:3037] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:00.960940 8669 raft_consensus.cc:504] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:00.961289 8669 leader_election.cc:288] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 election: Requested vote from peers ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935), eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787)
I20260925 00:43:00.961704 8594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50a71d7786b4355b446842911e5cf5e" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac"
I20260925 00:43:00.961845 8594 raft_consensus.cc:3037] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:00.962846 8454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50a71d7786b4355b446842911e5cf5e" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca961619ac6547939cb7a50a8b68db59"
I20260925 00:43:00.962986 8454 raft_consensus.cc:3037] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:00.963307 8594 raft_consensus.cc:2454] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 1.
I20260925 00:43:00.963601 8246 leader_election.cc:302] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d7c45c8384e4b07938ffa7a97f33a0f, eaf3cf8f75fb41ac8192cc5904cca3ac; no voters:
I20260925 00:43:00.963794 8669 raft_consensus.cc:2781] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:00.964048 8669 raft_consensus.cc:686] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 LEADER]: Becoming Leader. State: Replica: 0d7c45c8384e4b07938ffa7a97f33a0f, State: Running, Role: LEADER
I20260925 00:43:00.964345 8454 raft_consensus.cc:2454] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 1.
I20260925 00:43:00.964469 8669 consensus_queue.cc:237] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:00.967944 8157 catalog_manager.cc:5495] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65). New cstate: current_term: 1 leader_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } health_report { overall_health: UNKNOWN } } }
Sep 25 00:43:01 dist-test-slave-w8nh krb5kdc[8112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296979, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
W20260925 00:43:01.021575 8641 tablet.cc:2369] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:43:01.023274 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.014056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.6.72.126:41533)
0925 00:43:01.014307 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:01.014314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:01.014494 (+ 180us) server_negotiation.cc:407] Connection header received
0925 00:43:01.014692 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:01.014697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:01.014742 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:01.014837 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:01.015637 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.016192 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.017040 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.017262 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.017389 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.019857 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:01.019860 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:01.019862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:01.019881 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:01.021307 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:01.021818 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:01.021822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:01.021824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:01.021919 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:01.022205 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:01.022208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:01.022209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:01.022430 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:01.022512 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:01.022968 (+ 456us) server_negotiation.cc:299] Negotiation successful
0925 00:43:01.023086 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260925 00:43:01.023288 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.014034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41533 (local address 127.0.0.1:34254)
0925 00:43:01.014393 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:01.014406 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:01.014563 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:01.014856 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:01.014858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:01.015393 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:01.015494 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.015502 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.016352 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.016358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.016910 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.016919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.017396 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.017400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.017509 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.018091 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:01.018107 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:01.019688 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:01.021416 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:01.021429 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:01.021436 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:01.021686 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:01.022022 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:01.022025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:01.022027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:01.022112 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:01.022602 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:01.022608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:01.022724 (+ 116us) client_negotiation.cc:769] Sending connection context
0925 00:43:01.022860 (+ 136us) client_negotiation.cc:240] Negotiation successful
0925 00:43:01.023010 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":5,"thread_start_us":124,"threads_started":1}
I20260925 00:43:01.027544 8157 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:34254:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 00:43:01.028337 8157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:01.033377 8294 tablet_service.cc:1467] Processing CreateTablet for tablet bcfc64371197494e85ad9f9571d91e7b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc987b6c657240ac9aee41708869bd3d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 00:43:01.033375 8574 tablet_service.cc:1467] Processing CreateTablet for tablet bcfc64371197494e85ad9f9571d91e7b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc987b6c657240ac9aee41708869bd3d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 00:43:01.033350 8434 tablet_service.cc:1467] Processing CreateTablet for tablet bcfc64371197494e85ad9f9571d91e7b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc987b6c657240ac9aee41708869bd3d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 00:43:01.034157 8294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcfc64371197494e85ad9f9571d91e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:01.034158 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcfc64371197494e85ad9f9571d91e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:01.034166 8434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcfc64371197494e85ad9f9571d91e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:01.037150 8665 tablet_bootstrap.cc:492] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: Bootstrap starting.
I20260925 00:43:01.037194 8664 tablet_bootstrap.cc:492] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: Bootstrap starting.
I20260925 00:43:01.038038 8664 tablet_bootstrap.cc:654] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:01.038055 8665 tablet_bootstrap.cc:654] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:01.038743 8663 tablet_bootstrap.cc:492] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: Bootstrap starting.
I20260925 00:43:01.039353 8665 tablet_bootstrap.cc:492] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: No bootstrap required, opened a new log
I20260925 00:43:01.039407 8664 tablet_bootstrap.cc:492] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: No bootstrap required, opened a new log
I20260925 00:43:01.039467 8665 ts_tablet_manager.cc:1397] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:01.039479 8664 ts_tablet_manager.cc:1397] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:01.039788 8663 tablet_bootstrap.cc:654] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:01.039942 8665 raft_consensus.cc:348] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.039966 8664 raft_consensus.cc:348] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.040063 8665 raft_consensus.cc:374] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:01.040098 8665 raft_consensus.cc:729] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf3cf8f75fb41ac8192cc5904cca3ac, State: Initialized, Role: FOLLOWER
I20260925 00:43:01.040102 8664 raft_consensus.cc:374] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:01.040141 8664 raft_consensus.cc:729] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca961619ac6547939cb7a50a8b68db59, State: Initialized, Role: FOLLOWER
I20260925 00:43:01.040261 8665 consensus_queue.cc:260] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.040316 8664 consensus_queue.cc:260] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.040570 8665 ts_tablet_manager.cc:1428] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:01.040613 8664 ts_tablet_manager.cc:1428] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 00:43:01.040580 8670 tablet_replica.cc:439] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } } }
I20260925 00:43:01.041540 8670 tablet_replica.cc:442] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:01.041558 8671 tablet_replica.cc:439] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } } }
I20260925 00:43:01.041677 8663 tablet_bootstrap.cc:492] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: No bootstrap required, opened a new log
I20260925 00:43:01.041703 8671 tablet_replica.cc:442] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:01.041766 8663 ts_tablet_manager.cc:1397] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:01.042323 8663 raft_consensus.cc:348] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.042488 8663 raft_consensus.cc:374] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:01.042526 8663 raft_consensus.cc:729] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d7c45c8384e4b07938ffa7a97f33a0f, State: Initialized, Role: FOLLOWER
I20260925 00:43:01.042673 8663 consensus_queue.cc:260] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.042932 8669 tablet_replica.cc:439] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } } }
I20260925 00:43:01.043054 8669 tablet_replica.cc:442] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:01.043612 8663 ts_tablet_manager.cc:1428] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260925 00:43:01.044068 8361 tablet.cc:2369] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:43:01.046958 8501 tablet.cc:2369] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:43:01.133950 8670 raft_consensus.cc:482] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:01.134106 8670 raft_consensus.cc:504] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.135653 8670 leader_election.cc:288] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65:38829), ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935)
I20260925 00:43:01.141019 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.136349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:43443 (local address 127.6.72.66:32935)
0925 00:43:01.136505 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:01.136511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:01.136527 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:01.136635 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:01.136637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:01.136676 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:01.136742 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:01.137019 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.137729 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.139466 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.140168 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.140333 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.140407 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:01.140772 (+ 365us) server_negotiation.cc:299] Negotiation successful
0925 00:43:01.140871 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 00:43:01.141043 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.136138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:53861 (local address 127.6.72.65:38829)
0925 00:43:01.136292 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:01.136296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:01.136311 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:43:01.136369 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:01.136371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:01.136446 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:01.136529 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:01.136852 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.137650 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.139466 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.140168 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.140333 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.140407 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:01.140772 (+ 365us) server_negotiation.cc:299] Negotiation successful
0925 00:43:01.140870 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 00:43:01.141050 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.135849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:38829 (local address 127.6.72.67:53861)
0925 00:43:01.135989 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:01.136008 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:01.136112 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:01.136551 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:01.136554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:01.136568 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:01.136707 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.136712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.137761 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.137764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.139298 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.139310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.140355 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.140361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.140557 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.140649 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 00:43:01.140731 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 00:43:01.140864 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":225}
I20260925 00:43:01.141069 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.135928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:32935 (local address 127.6.72.67:43443)
0925 00:43:01.136423 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:01.136436 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:01.136551 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:01.136776 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:01.136780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:01.136800 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:01.136921 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.136925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.137828 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.137830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.139298 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.139310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.140363 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.140366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.140558 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.140649 (+ 91us) client_negotiation.cc:769] Sending connection context
0925 00:43:01.140731 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 00:43:01.140864 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":165,"thread_start_us":131,"threads_started":1}
I20260925 00:43:01.141924 8310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcfc64371197494e85ad9f9571d91e7b" candidate_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" is_pre_election: true
I20260925 00:43:01.142119 8454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcfc64371197494e85ad9f9571d91e7b" candidate_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca961619ac6547939cb7a50a8b68db59" is_pre_election: true
I20260925 00:43:01.142253 8310 raft_consensus.cc:2454] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaf3cf8f75fb41ac8192cc5904cca3ac in term 0.
I20260925 00:43:01.142294 8454 raft_consensus.cc:2454] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaf3cf8f75fb41ac8192cc5904cca3ac in term 0.
I20260925 00:43:01.142630 8526 leader_election.cc:302] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca961619ac6547939cb7a50a8b68db59, eaf3cf8f75fb41ac8192cc5904cca3ac; no voters:
I20260925 00:43:01.142948 8670 raft_consensus.cc:2781] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:01.143020 8670 raft_consensus.cc:482] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:01.143072 8670 raft_consensus.cc:3037] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:01.144218 8670 raft_consensus.cc:504] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.144757 8670 leader_election.cc:288] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [CANDIDATE]: Term 1 election: Requested vote from peers 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65:38829), ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935)
I20260925 00:43:01.145126 8310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcfc64371197494e85ad9f9571d91e7b" candidate_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f"
I20260925 00:43:01.145181 8454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcfc64371197494e85ad9f9571d91e7b" candidate_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca961619ac6547939cb7a50a8b68db59"
I20260925 00:43:01.145252 8310 raft_consensus.cc:3037] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:01.145327 8454 raft_consensus.cc:3037] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:01.146332 8454 raft_consensus.cc:2454] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf3cf8f75fb41ac8192cc5904cca3ac in term 1.
I20260925 00:43:01.146360 8310 raft_consensus.cc:2454] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf3cf8f75fb41ac8192cc5904cca3ac in term 1.
I20260925 00:43:01.146613 8526 leader_election.cc:302] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca961619ac6547939cb7a50a8b68db59, eaf3cf8f75fb41ac8192cc5904cca3ac; no voters:
I20260925 00:43:01.146795 8670 raft_consensus.cc:2781] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:01.147045 8670 raft_consensus.cc:686] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 LEADER]: Becoming Leader. State: Replica: eaf3cf8f75fb41ac8192cc5904cca3ac, State: Running, Role: LEADER
I20260925 00:43:01.147434 8670 consensus_queue.cc:237] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.149930 8686 tablet_replica.cc:439] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: TxnStatusTablet state changed. Reason: New leader eaf3cf8f75fb41ac8192cc5904cca3ac. Latest consensus state: current_term: 1 leader_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } } }
I20260925 00:43:01.150055 8686 tablet_replica.cc:442] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: This TxnStatusTablet replica's current role is: LEADER
I20260925 00:43:01.150342 8688 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 00:43:01.150622 8688 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 00:43:01.151059 8157 catalog_manager.cc:5495] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac reported cstate change: term changed from 0 to 1, leader changed from <none> to eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67). New cstate: current_term: 1 leader_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } health_report { overall_health: HEALTHY } } }
I20260925 00:43:01.170673 8662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.166160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.6.72.67:45787)
0925 00:43:01.166269 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:01.166273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:01.166286 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:43:01.166521 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:01.166524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:01.166581 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:01.166689 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:01.167002 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.167669 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.168962 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.169192 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.169310 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.169684 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:01.170222 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:01.170328 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:01.170444 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 00:43:01.170501 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 00:43:01.170661 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.166012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:45787 (local address 127.0.0.1:56826)
0925 00:43:01.166204 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:01.166220 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:01.166337 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:01.166728 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:01.166731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:01.166739 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:43:01.166853 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.166859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.167787 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.167791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.168813 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.168825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.169338 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.169342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.169435 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.169497 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:43:01.170326 (+ 829us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:43:01.170342 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 00:43:01.170415 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 00:43:01.170509 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260925 00:43:01.176749 8310 raft_consensus.cc:1261] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Refusing update from remote peer eaf3cf8f75fb41ac8192cc5904cca3ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:01.176750 8454 raft_consensus.cc:1261] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Refusing update from remote peer eaf3cf8f75fb41ac8192cc5904cca3ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:01.177462 8670 consensus_queue.cc:1035] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:01.177646 8686 consensus_queue.cc:1035] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:01.180707 8669 tablet_replica.cc:439] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: TxnStatusTablet state changed. Reason: New leader eaf3cf8f75fb41ac8192cc5904cca3ac. Latest consensus state: current_term: 1 leader_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } } }
I20260925 00:43:01.180824 8669 tablet_replica.cc:442] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:01.181723 8671 tablet_replica.cc:439] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: TxnStatusTablet state changed. Reason: New leader eaf3cf8f75fb41ac8192cc5904cca3ac. Latest consensus state: current_term: 1 leader_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } } }
I20260925 00:43:01.181887 8671 tablet_replica.cc:442] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:01.185977 8669 tablet_replica.cc:439] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } } }
I20260925 00:43:01.186020 8671 tablet_replica.cc:439] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } } }
I20260925 00:43:01.186097 8669 tablet_replica.cc:442] T bcfc64371197494e85ad9f9571d91e7b P 0d7c45c8384e4b07938ffa7a97f33a0f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:01.186128 8671 tablet_replica.cc:442] T bcfc64371197494e85ad9f9571d91e7b P ca961619ac6547939cb7a50a8b68db59: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:01.186625 8696 tablet_replica.cc:439] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } } }
I20260925 00:43:01.186825 8696 tablet_replica.cc:442] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: This TxnStatusTablet replica's current role is: LEADER
I20260925 00:43:01.187037 8693 mvcc.cc:205] Tried to move back new op lower bound from 7333056434892398592 to 7333056434782629888. Current Snapshot: MvccSnapshot[applied={T|T < 7333056434892398592}]
I20260925 00:43:01.187053 8670 tablet_replica.cc:439] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } } }
I20260925 00:43:01.187150 8670 tablet_replica.cc:442] T bcfc64371197494e85ad9f9571d91e7b P eaf3cf8f75fb41ac8192cc5904cca3ac: This TxnStatusTablet replica's current role is: LEADER
I20260925 00:43:01.197531 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.193799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.6.72.65:38829)
0925 00:43:01.194108 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:01.194112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:01.194122 (+ 10us) server_negotiation.cc:407] Connection header received
0925 00:43:01.194189 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:01.194191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:01.194239 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:01.194352 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:01.194617 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.195135 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.195992 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.196230 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.196388 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.196693 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:01.197109 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:01.197246 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:01.197309 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 00:43:01.197367 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240}
I20260925 00:43:01.209352 8662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.206135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:49731 (local address 127.6.72.67:45787)
0925 00:43:01.206297 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:01.206302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:01.206322 (+ 20us) server_negotiation.cc:407] Connection header received
0925 00:43:01.206368 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:01.206370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:01.206423 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:01.206510 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:01.206793 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.207346 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.208225 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.208478 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.208593 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.208908 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:01.208983 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:01.209061 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:01.209154 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 00:43:01.209203 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 00:43:01.209370 8703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.205646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:45787 (local address 127.6.72.65:49731)
0925 00:43:01.205972 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:01.205987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:01.206107 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:01.206522 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:01.206524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:01.206533 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:43:01.206680 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.206686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.207463 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.207470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.208100 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.208108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.208591 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.208594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.208704 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.208746 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:43:01.209065 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:43:01.209073 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 00:43:01.209140 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 00:43:01.209217 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":153,"threads_started":1}
I20260925 00:43:01.217036 8454 raft_consensus.cc:1261] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Refusing update from remote peer 0d7c45c8384e4b07938ffa7a97f33a0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:01.217133 8594 raft_consensus.cc:1261] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Refusing update from remote peer 0d7c45c8384e4b07938ffa7a97f33a0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:01.217542 8669 consensus_queue.cc:1035] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:01.217728 8674 consensus_queue.cc:1035] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:01.243655 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.239416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:38829 (local address 127.6.72.67:40113)
0925 00:43:01.239571 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:01.239587 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:01.239680 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:01.239946 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:01.239949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:01.239956 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:43:01.240077 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.240088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.241292 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.241304 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.242277 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.242327 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.242886 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.242890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.242989 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.243051 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:43:01.243344 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:43:01.243350 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 00:43:01.243416 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 00:43:01.243500 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 00:43:01.243664 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.239490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:40113 (local address 127.6.72.65:38829)
0925 00:43:01.239607 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:01.239614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:01.239639 (+ 25us) server_negotiation.cc:407] Connection header received
0925 00:43:01.239791 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:01.239794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:01.239853 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:01.239935 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:01.240209 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.241125 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.242429 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.242765 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.242861 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.243185 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:01.243257 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:01.243312 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:01.243432 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 00:43:01.243523 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 00:43:01.272480 8157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:01.272689 8157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:01.275632 8157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1: Sending DeleteTablet for 3 replicas of tablet b50a71d7786b4355b446842911e5cf5e
I20260925 00:43:01.276525 8434 tablet_service.cc:1514] Processing DeleteTablet for tablet b50a71d7786b4355b446842911e5cf5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:01 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:32966
I20260925 00:43:01.276566 8574 tablet_service.cc:1514] Processing DeleteTablet for tablet b50a71d7786b4355b446842911e5cf5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:01 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:56824
I20260925 00:43:01.277175 8710 tablet_replica.cc:330] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: stopping tablet replica
I20260925 00:43:01.277374 8294 tablet_service.cc:1514] Processing DeleteTablet for tablet b50a71d7786b4355b446842911e5cf5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:01 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:44226
I20260925 00:43:01.277616 8710 raft_consensus.cc:2229] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:01.277891 8710 raft_consensus.cc:2258] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:01.278612 8157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:01.279085 8157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:01.279812 8710 ts_tablet_manager.cc:1905] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:01.282779 8712 tablet_replica.cc:330] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: stopping tablet replica
I20260925 00:43:01.283134 8712 raft_consensus.cc:2229] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:01.283548 8710 ts_tablet_manager.cc:1918] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:01.283558 8712 raft_consensus.cc:2258] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:01.283718 8710 log.cc:1196] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/b50a71d7786b4355b446842911e5cf5e
I20260925 00:43:01.284353 8710 ts_tablet_manager.cc:1939] T b50a71d7786b4355b446842911e5cf5e P ca961619ac6547939cb7a50a8b68db59: Deleting consensus metadata
I20260925 00:43:01.284889 8294 tablet_service.cc:1467] Processing CreateTablet for tablet 8394cfbaf1e245af87c69deea4b14ab5 (DEFAULT_TABLE table=test-table [id=233e3ece6aa64d6aaa9ba1bcc96d088a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:01.285207 8294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8394cfbaf1e245af87c69deea4b14ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:01.285308 8434 tablet_service.cc:1467] Processing CreateTablet for tablet 8394cfbaf1e245af87c69deea4b14ab5 (DEFAULT_TABLE table=test-table [id=233e3ece6aa64d6aaa9ba1bcc96d088a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:01.285552 8434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8394cfbaf1e245af87c69deea4b14ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:01.287403 8574 tablet_service.cc:1467] Processing CreateTablet for tablet 8394cfbaf1e245af87c69deea4b14ab5 (DEFAULT_TABLE table=test-table [id=233e3ece6aa64d6aaa9ba1bcc96d088a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:01.287678 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8394cfbaf1e245af87c69deea4b14ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:01.287907 8664 tablet_bootstrap.cc:492] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59: Bootstrap starting.
I20260925 00:43:01.289397 8664 tablet_bootstrap.cc:654] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:01.290091 8665 tablet_bootstrap.cc:492] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Bootstrap starting.
I20260925 00:43:01.290912 8142 catalog_manager.cc:4841] TS ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935): tablet b50a71d7786b4355b446842911e5cf5e (table test-table [id=58b916a9bc9c4e69948794d945129dca]) successfully deleted
I20260925 00:43:01.291451 8665 tablet_bootstrap.cc:654] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:01.291975 8664 tablet_bootstrap.cc:492] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59: No bootstrap required, opened a new log
I20260925 00:43:01.292055 8664 ts_tablet_manager.cc:1397] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260925 00:43:01.292604 8664 raft_consensus.cc:348] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.292725 8664 raft_consensus.cc:374] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:01.292755 8664 raft_consensus.cc:729] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca961619ac6547939cb7a50a8b68db59, State: Initialized, Role: FOLLOWER
I20260925 00:43:01.292870 8664 consensus_queue.cc:260] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.293088 8664 ts_tablet_manager.cc:1428] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:01.293519 8663 tablet_bootstrap.cc:492] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Bootstrap starting.
I20260925 00:43:01.294498 8663 tablet_bootstrap.cc:654] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:01.296788 8712 ts_tablet_manager.cc:1905] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:01.297260 8711 tablet_replica.cc:330] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: stopping tablet replica
I20260925 00:43:01.297955 8711 raft_consensus.cc:2229] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:01.298164 8711 raft_consensus.cc:2258] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:01.301644 8665 tablet_bootstrap.cc:492] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac: No bootstrap required, opened a new log
I20260925 00:43:01.301743 8665 ts_tablet_manager.cc:1397] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260925 00:43:01.302217 8665 raft_consensus.cc:348] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.302383 8665 raft_consensus.cc:374] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:01.302423 8665 raft_consensus.cc:729] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf3cf8f75fb41ac8192cc5904cca3ac, State: Initialized, Role: FOLLOWER
I20260925 00:43:01.302551 8665 consensus_queue.cc:260] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.302809 8665 ts_tablet_manager.cc:1428] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:01.302958 8711 ts_tablet_manager.cc:1905] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:01.306407 8711 ts_tablet_manager.cc:1918] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:01.306522 8711 log.cc:1196] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/b50a71d7786b4355b446842911e5cf5e
I20260925 00:43:01.306792 8712 ts_tablet_manager.cc:1918] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:01.306895 8712 log.cc:1196] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/b50a71d7786b4355b446842911e5cf5e
I20260925 00:43:01.306890 8711 ts_tablet_manager.cc:1939] T b50a71d7786b4355b446842911e5cf5e P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting consensus metadata
I20260925 00:43:01.307212 8712 ts_tablet_manager.cc:1939] T b50a71d7786b4355b446842911e5cf5e P 0d7c45c8384e4b07938ffa7a97f33a0f: Deleting consensus metadata
I20260925 00:43:01.308151 8142 catalog_manager.cc:4841] TS eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787): tablet b50a71d7786b4355b446842911e5cf5e (table test-table [id=58b916a9bc9c4e69948794d945129dca]) successfully deleted
I20260925 00:43:01.308329 8144 catalog_manager.cc:4841] TS 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65:38829): tablet b50a71d7786b4355b446842911e5cf5e (table test-table [id=58b916a9bc9c4e69948794d945129dca]) successfully deleted
I20260925 00:43:01.309959 8663 tablet_bootstrap.cc:492] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f: No bootstrap required, opened a new log
I20260925 00:43:01.310055 8663 ts_tablet_manager.cc:1397] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260925 00:43:01.310513 8663 raft_consensus.cc:348] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.310633 8663 raft_consensus.cc:374] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:01.310669 8663 raft_consensus.cc:729] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d7c45c8384e4b07938ffa7a97f33a0f, State: Initialized, Role: FOLLOWER
I20260925 00:43:01.310809 8663 consensus_queue.cc:260] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.311090 8663 ts_tablet_manager.cc:1428] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:01.414022 8674 raft_consensus.cc:482] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:01.414135 8671 raft_consensus.cc:482] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:01.414224 8674 raft_consensus.cc:504] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.414284 8671 raft_consensus.cc:504] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.414760 8674 leader_election.cc:288] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935), eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787)
I20260925 00:43:01.415172 8454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8394cfbaf1e245af87c69deea4b14ab5" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca961619ac6547939cb7a50a8b68db59" is_pre_election: true
I20260925 00:43:01.415304 8454 raft_consensus.cc:2454] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 0.
I20260925 00:43:01.415512 8594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8394cfbaf1e245af87c69deea4b14ab5" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" is_pre_election: true
I20260925 00:43:01.415596 8671 leader_election.cc:288] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65:38829), eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787)
I20260925 00:43:01.415654 8594 raft_consensus.cc:2454] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 0.
I20260925 00:43:01.415642 8246 leader_election.cc:302] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d7c45c8384e4b07938ffa7a97f33a0f, ca961619ac6547939cb7a50a8b68db59; no voters:
I20260925 00:43:01.415808 8674 raft_consensus.cc:2781] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:01.415853 8674 raft_consensus.cc:482] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:01.415889 8674 raft_consensus.cc:3037] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:01.417438 8674 raft_consensus.cc:504] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.417809 8674 leader_election.cc:288] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 election: Requested vote from peers ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935), eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787)
I20260925 00:43:01.418237 8594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8394cfbaf1e245af87c69deea4b14ab5" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac"
I20260925 00:43:01.418354 8594 raft_consensus.cc:3037] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:01.419552 8594 raft_consensus.cc:2454] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 1.
I20260925 00:43:01.420121 8454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8394cfbaf1e245af87c69deea4b14ab5" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca961619ac6547939cb7a50a8b68db59"
I20260925 00:43:01.420307 8454 raft_consensus.cc:3037] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:01.421243 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.416548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:34559 (local address 127.6.72.65:38829)
0925 00:43:01.416658 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:01.416662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:01.416988 (+ 326us) server_negotiation.cc:407] Connection header received
0925 00:43:01.417206 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:01.417209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:01.417255 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:01.417298 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:01.417640 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.418398 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.419933 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.420563 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.420637 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.420699 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:01.421047 (+ 348us) server_negotiation.cc:299] Negotiation successful
0925 00:43:01.421118 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 00:43:01.421479 8454 raft_consensus.cc:2454] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 1.
I20260925 00:43:01.421644 8716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.416189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:38829 (local address 127.6.72.66:34559)
0925 00:43:01.416904 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:01.416918 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:01.417065 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:01.417427 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:01.417430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:01.417444 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:01.417538 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.417542 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.418537 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.418540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.419796 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.419806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.420787 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.420790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.420919 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.420940 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 00:43:01.421411 (+ 471us) client_negotiation.cc:240] Negotiation successful
0925 00:43:01.421497 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":1,"thread_start_us":116,"threads_started":1}
I20260925 00:43:01.421815 8246 leader_election.cc:302] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d7c45c8384e4b07938ffa7a97f33a0f, ca961619ac6547939cb7a50a8b68db59; no voters:
I20260925 00:43:01.422008 8310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8394cfbaf1e245af87c69deea4b14ab5" candidate_uuid: "ca961619ac6547939cb7a50a8b68db59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" is_pre_election: true
I20260925 00:43:01.422079 8674 raft_consensus.cc:2781] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:01.422479 8674 raft_consensus.cc:686] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 LEADER]: Becoming Leader. State: Replica: 0d7c45c8384e4b07938ffa7a97f33a0f, State: Running, Role: LEADER
I20260925 00:43:01.422670 8662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.416658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:42745 (local address 127.6.72.67:45787)
0925 00:43:01.416764 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:01.416768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:01.417087 (+ 319us) server_negotiation.cc:407] Connection header received
0925 00:43:01.417300 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:01.417306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:01.417366 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:01.417419 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:01.417963 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.418682 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.420876 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.421500 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.421581 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.421651 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:01.422480 (+ 829us) server_negotiation.cc:299] Negotiation successful
0925 00:43:01.422566 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 00:43:01.422640 8674 consensus_queue.cc:237] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:01.423266 8717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:01.416325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:45787 (local address 127.6.72.66:42745)
0925 00:43:01.417001 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:01.417016 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:01.417161 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:01.417635 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:01.417638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:01.417668 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:01.417784 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.417791 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.419232 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.419238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.420719 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:01.420731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:01.422246 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:01.422252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:01.422345 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:01.422366 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 00:43:01.423015 (+ 649us) client_negotiation.cc:240] Negotiation successful
0925 00:43:01.423109 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":113,"threads_started":1}
I20260925 00:43:01.424136 8594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8394cfbaf1e245af87c69deea4b14ab5" candidate_uuid: "ca961619ac6547939cb7a50a8b68db59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" is_pre_election: true
I20260925 00:43:01.424342 8594 raft_consensus.cc:2379] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca961619ac6547939cb7a50a8b68db59 in current term 1: Already voted for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in this term.
I20260925 00:43:01.424650 8386 leader_election.cc:302] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca961619ac6547939cb7a50a8b68db59; no voters: 0d7c45c8384e4b07938ffa7a97f33a0f, eaf3cf8f75fb41ac8192cc5904cca3ac
I20260925 00:43:01.424917 8671 raft_consensus.cc:2726] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 00:43:01.424885 8155 catalog_manager.cc:5495] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65). New cstate: current_term: 1 leader_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260925 00:43:01.467758 8594 raft_consensus.cc:1261] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Refusing update from remote peer 0d7c45c8384e4b07938ffa7a97f33a0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:01.467787 8454 raft_consensus.cc:1261] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Refusing update from remote peer 0d7c45c8384e4b07938ffa7a97f33a0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:01.468146 8669 consensus_queue.cc:1035] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:01.468323 8674 consensus_queue.cc:1035] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:01.471652 8691 mvcc.cc:205] Tried to move back new op lower bound from 7333056436088160256 to 7333056435908968448. Current Snapshot: MvccSnapshot[applied={T|T < 7333056436088160256}]
W20260925 00:43:01.481992 8693 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 00:43:03.031817 8155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:03.031962 8155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:03.034190 8155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1: Sending DeleteTablet for 3 replicas of tablet 8394cfbaf1e245af87c69deea4b14ab5
I20260925 00:43:03.034806 8294 tablet_service.cc:1514] Processing DeleteTablet for tablet 8394cfbaf1e245af87c69deea4b14ab5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:03 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:44226
I20260925 00:43:03.034806 8574 tablet_service.cc:1514] Processing DeleteTablet for tablet 8394cfbaf1e245af87c69deea4b14ab5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:03 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:56824
I20260925 00:43:03.035086 8434 tablet_service.cc:1514] Processing DeleteTablet for tablet 8394cfbaf1e245af87c69deea4b14ab5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:03 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:32966
I20260925 00:43:03.035122 8745 tablet_replica.cc:330] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f: stopping tablet replica
I20260925 00:43:03.035230 8745 raft_consensus.cc:2229] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:03.035476 8745 raft_consensus.cc:2258] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:03.035760 8744 tablet_replica.cc:330] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac: stopping tablet replica
I20260925 00:43:03.035895 8744 raft_consensus.cc:2229] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:03.036003 8744 raft_consensus.cc:2258] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:03.036126 8745 ts_tablet_manager.cc:1905] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:03.036588 8744 ts_tablet_manager.cc:1905] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:03.038800 8745 ts_tablet_manager.cc:1918] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 00:43:03.038882 8745 log.cc:1196] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/8394cfbaf1e245af87c69deea4b14ab5
I20260925 00:43:03.039163 8745 ts_tablet_manager.cc:1939] T 8394cfbaf1e245af87c69deea4b14ab5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Deleting consensus metadata
I20260925 00:43:03.039168 8744 ts_tablet_manager.cc:1918] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 00:43:03.039239 8744 log.cc:1196] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/8394cfbaf1e245af87c69deea4b14ab5
I20260925 00:43:03.039472 8744 ts_tablet_manager.cc:1939] T 8394cfbaf1e245af87c69deea4b14ab5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting consensus metadata
I20260925 00:43:03.039614 8746 tablet_replica.cc:330] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59: stopping tablet replica
I20260925 00:43:03.039726 8746 raft_consensus.cc:2229] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:03.039813 8746 raft_consensus.cc:2258] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:03.040412 8746 ts_tablet_manager.cc:1905] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:03.040558 8142 catalog_manager.cc:4841] TS eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787): tablet 8394cfbaf1e245af87c69deea4b14ab5 (table test-table [id=233e3ece6aa64d6aaa9ba1bcc96d088a]) successfully deleted
I20260925 00:43:03.040583 8144 catalog_manager.cc:4841] TS 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65:38829): tablet 8394cfbaf1e245af87c69deea4b14ab5 (table test-table [id=233e3ece6aa64d6aaa9ba1bcc96d088a]) successfully deleted
I20260925 00:43:03.042838 8746 ts_tablet_manager.cc:1918] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 00:43:03.042903 8746 log.cc:1196] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/8394cfbaf1e245af87c69deea4b14ab5
I20260925 00:43:03.043130 8746 ts_tablet_manager.cc:1939] T 8394cfbaf1e245af87c69deea4b14ab5 P ca961619ac6547939cb7a50a8b68db59: Deleting consensus metadata
I20260925 00:43:03.043787 8142 catalog_manager.cc:4841] TS ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935): tablet 8394cfbaf1e245af87c69deea4b14ab5 (table test-table [id=233e3ece6aa64d6aaa9ba1bcc96d088a]) successfully deleted
Sep 25 00:43:03 dist-test-slave-w8nh krb5kdc[8112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296983, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 00:43:03 dist-test-slave-w8nh krb5kdc[8112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296983, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:03.069029 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:03.058409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34256 (local address 127.6.72.126:41533)
0925 00:43:03.058688 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:03.058695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:03.058759 (+ 64us) server_negotiation.cc:407] Connection header received
0925 00:43:03.058806 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:03.058809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:03.058855 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:03.058933 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:03.059836 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.060544 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.061841 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.062097 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.062191 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:03.065225 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:03.065229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:03.065232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:03.065282 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:03.066979 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:03.067545 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:03.067548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:03.067549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:03.067665 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:03.067963 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:03.067966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:03.067967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:03.068196 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:03.068441 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:03.068753 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 00:43:03.068883 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":119,"threads_started":1}
I20260925 00:43:03.072603 8155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:03.073127 8155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:03.079259 8294 tablet_service.cc:1467] Processing CreateTablet for tablet 020e8cbbe37540789e5b4dc3b31377e5 (DEFAULT_TABLE table=test-table [id=3631b79cbd7041f58bba32ceaa4ba6a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:03.079677 8294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020e8cbbe37540789e5b4dc3b31377e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:03.079627 8574 tablet_service.cc:1467] Processing CreateTablet for tablet 020e8cbbe37540789e5b4dc3b31377e5 (DEFAULT_TABLE table=test-table [id=3631b79cbd7041f58bba32ceaa4ba6a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:03.079736 8434 tablet_service.cc:1467] Processing CreateTablet for tablet 020e8cbbe37540789e5b4dc3b31377e5 (DEFAULT_TABLE table=test-table [id=3631b79cbd7041f58bba32ceaa4ba6a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:03.079881 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020e8cbbe37540789e5b4dc3b31377e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:03.080045 8434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 020e8cbbe37540789e5b4dc3b31377e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:03.082402 8755 tablet_bootstrap.cc:492] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59: Bootstrap starting.
I20260925 00:43:03.082723 8756 tablet_bootstrap.cc:492] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Bootstrap starting.
I20260925 00:43:03.083252 8757 tablet_bootstrap.cc:492] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Bootstrap starting.
I20260925 00:43:03.083658 8756 tablet_bootstrap.cc:654] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:03.083706 8755 tablet_bootstrap.cc:654] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:03.084374 8757 tablet_bootstrap.cc:654] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:03.085083 8756 tablet_bootstrap.cc:492] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac: No bootstrap required, opened a new log
I20260925 00:43:03.085211 8756 ts_tablet_manager.cc:1397] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:03.085427 8755 tablet_bootstrap.cc:492] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59: No bootstrap required, opened a new log
I20260925 00:43:03.085513 8755 ts_tablet_manager.cc:1397] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260925 00:43:03.085741 8757 tablet_bootstrap.cc:492] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f: No bootstrap required, opened a new log
I20260925 00:43:03.085722 8756 raft_consensus.cc:348] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.085824 8757 ts_tablet_manager.cc:1397] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260925 00:43:03.085845 8756 raft_consensus.cc:374] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:03.085882 8756 raft_consensus.cc:729] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf3cf8f75fb41ac8192cc5904cca3ac, State: Initialized, Role: FOLLOWER
I20260925 00:43:03.085978 8755 raft_consensus.cc:348] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.086028 8756 consensus_queue.cc:260] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.086083 8755 raft_consensus.cc:374] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:03.086119 8755 raft_consensus.cc:729] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca961619ac6547939cb7a50a8b68db59, State: Initialized, Role: FOLLOWER
I20260925 00:43:03.086246 8755 consensus_queue.cc:260] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.086349 8756 ts_tablet_manager.cc:1428] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:03.086408 8757 raft_consensus.cc:348] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.086527 8757 raft_consensus.cc:374] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:03.086561 8757 raft_consensus.cc:729] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d7c45c8384e4b07938ffa7a97f33a0f, State: Initialized, Role: FOLLOWER
I20260925 00:43:03.086674 8755 ts_tablet_manager.cc:1428] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 00:43:03.086756 8757 consensus_queue.cc:260] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.087072 8757 ts_tablet_manager.cc:1428] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 00:43:03.174175 8761 raft_consensus.cc:482] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:03.174422 8761 raft_consensus.cc:504] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.175009 8761 leader_election.cc:288] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65:38829), eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787)
I20260925 00:43:03.175493 8310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020e8cbbe37540789e5b4dc3b31377e5" candidate_uuid: "ca961619ac6547939cb7a50a8b68db59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" is_pre_election: true
I20260925 00:43:03.175493 8594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020e8cbbe37540789e5b4dc3b31377e5" candidate_uuid: "ca961619ac6547939cb7a50a8b68db59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" is_pre_election: true
I20260925 00:43:03.175654 8594 raft_consensus.cc:2454] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca961619ac6547939cb7a50a8b68db59 in term 0.
I20260925 00:43:03.175654 8310 raft_consensus.cc:2454] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca961619ac6547939cb7a50a8b68db59 in term 0.
I20260925 00:43:03.176025 8388 leader_election.cc:302] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0d7c45c8384e4b07938ffa7a97f33a0f, ca961619ac6547939cb7a50a8b68db59, eaf3cf8f75fb41ac8192cc5904cca3ac; no voters:
I20260925 00:43:03.176268 8761 raft_consensus.cc:2781] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:03.176344 8761 raft_consensus.cc:482] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:03.176400 8761 raft_consensus.cc:3037] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:03.177419 8761 raft_consensus.cc:504] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.177827 8761 leader_election.cc:288] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [CANDIDATE]: Term 1 election: Requested vote from peers 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65:38829), eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787)
I20260925 00:43:03.178504 8594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020e8cbbe37540789e5b4dc3b31377e5" candidate_uuid: "ca961619ac6547939cb7a50a8b68db59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac"
I20260925 00:43:03.178504 8310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "020e8cbbe37540789e5b4dc3b31377e5" candidate_uuid: "ca961619ac6547939cb7a50a8b68db59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f"
I20260925 00:43:03.178663 8310 raft_consensus.cc:3037] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:03.178668 8594 raft_consensus.cc:3037] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:03.179912 8310 raft_consensus.cc:2454] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca961619ac6547939cb7a50a8b68db59 in term 1.
I20260925 00:43:03.179912 8594 raft_consensus.cc:2454] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca961619ac6547939cb7a50a8b68db59 in term 1.
I20260925 00:43:03.180301 8386 leader_election.cc:302] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca961619ac6547939cb7a50a8b68db59, eaf3cf8f75fb41ac8192cc5904cca3ac; no voters:
I20260925 00:43:03.180540 8761 raft_consensus.cc:2781] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:03.180764 8761 raft_consensus.cc:686] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 1 LEADER]: Becoming Leader. State: Replica: ca961619ac6547939cb7a50a8b68db59, State: Running, Role: LEADER
I20260925 00:43:03.181062 8761 consensus_queue.cc:237] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.183590 8157 catalog_manager.cc:5495] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca961619ac6547939cb7a50a8b68db59 (127.6.72.66). New cstate: current_term: 1 leader_uuid: "ca961619ac6547939cb7a50a8b68db59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260925 00:43:03.222647 8765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:03.218847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32974 (local address 127.6.72.66:32935)
0925 00:43:03.219131 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:03.219140 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:03.219152 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:43:03.219242 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:03.219244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:03.219296 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:03.219430 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:03.219695 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.220210 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.221107 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.221340 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.221449 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:03.221742 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:03.222195 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:03.222345 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:03.222421 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 00:43:03.222476 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":110,"threads_started":1}
I20260925 00:43:03.232568 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:03.229136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:45787 (local address 127.6.72.66:40353)
0925 00:43:03.229436 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:03.229453 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:03.229548 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:03.229798 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:03.229801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:03.229809 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:43:03.229927 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:03.229932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.230663 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.230668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:03.231292 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:03.231301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.231764 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.231767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:03.231861 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:03.231901 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:43:03.232190 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:43:03.232206 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 00:43:03.232335 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 00:43:03.232413 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":101,"threads_started":1}
I20260925 00:43:03.232556 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:03.229274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:40353 (local address 127.6.72.67:45787)
0925 00:43:03.229530 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:03.229539 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:03.229555 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:03.229642 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:03.229645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:03.229696 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:03.229791 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:03.230031 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.230556 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.231413 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.231650 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.231766 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:03.232040 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:03.232108 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:03.232186 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:03.232352 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 00:43:03.232400 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":83,"threads_started":1}
I20260925 00:43:03.238967 8594 raft_consensus.cc:1261] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Refusing update from remote peer ca961619ac6547939cb7a50a8b68db59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:03.239025 8310 raft_consensus.cc:1261] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Refusing update from remote peer ca961619ac6547939cb7a50a8b68db59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:03.239378 8761 consensus_queue.cc:1035] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:03.239562 8762 consensus_queue.cc:1035] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:03.242710 8726 mvcc.cc:205] Tried to move back new op lower bound from 7333056443342725120 to 7333056443112153088. Current Snapshot: MvccSnapshot[applied={T|T < 7333056443342725120}]
I20260925 00:43:03.258291 8765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:03.254585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:45749 (local address 127.6.72.66:32935)
0925 00:43:03.254713 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:03.254720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:03.254737 (+ 17us) server_negotiation.cc:407] Connection header received
0925 00:43:03.254871 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:03.254874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:03.254939 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:03.255033 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:03.255340 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.255858 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.256917 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.257186 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.257304 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:03.257621 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:03.257710 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:03.257787 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:03.258047 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 00:43:03.258119 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 00:43:03.258312 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:03.254291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:32935 (local address 127.6.72.67:45749)
0925 00:43:03.254615 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:03.254628 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:03.254745 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:03.255042 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:03.255045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:03.255052 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:43:03.255224 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:03.255230 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.255987 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.255990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:03.256811 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:03.256823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.257320 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.257325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:03.257435 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:03.257477 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:43:03.257928 (+ 451us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:43:03.257937 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 00:43:03.258011 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 00:43:03.258139 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":139,"threads_started":1}
I20260925 00:43:03.284384 8157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:03.284582 8157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:03.286973 8157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1: Sending DeleteTablet for 3 replicas of tablet 020e8cbbe37540789e5b4dc3b31377e5
I20260925 00:43:03.287813 8294 tablet_service.cc:1514] Processing DeleteTablet for tablet 020e8cbbe37540789e5b4dc3b31377e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:03 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:44226
I20260925 00:43:03.287832 8434 tablet_service.cc:1514] Processing DeleteTablet for tablet 020e8cbbe37540789e5b4dc3b31377e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:03 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:32966
I20260925 00:43:03.287853 8574 tablet_service.cc:1514] Processing DeleteTablet for tablet 020e8cbbe37540789e5b4dc3b31377e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:03 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:56824
I20260925 00:43:03.288053 8746 tablet_replica.cc:330] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59: stopping tablet replica
I20260925 00:43:03.288092 8744 tablet_replica.cc:330] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac: stopping tablet replica
I20260925 00:43:03.288196 8746 raft_consensus.cc:2229] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:03.288198 8744 raft_consensus.cc:2229] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:03.288342 8744 raft_consensus.cc:2258] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:03.288563 8746 raft_consensus.cc:2258] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:03.288765 8745 tablet_replica.cc:330] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f: stopping tablet replica
I20260925 00:43:03.288897 8745 raft_consensus.cc:2229] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:03.289009 8745 raft_consensus.cc:2258] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:03.289192 8744 ts_tablet_manager.cc:1905] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:03.290319 8157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:03.290849 8157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:03.291687 8746 ts_tablet_manager.cc:1905] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:03.292397 8744 ts_tablet_manager.cc:1918] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:03.292470 8744 log.cc:1196] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/020e8cbbe37540789e5b4dc3b31377e5
I20260925 00:43:03.294687 8746 ts_tablet_manager.cc:1918] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:03.294790 8746 log.cc:1196] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/020e8cbbe37540789e5b4dc3b31377e5
I20260925 00:43:03.295042 8746 ts_tablet_manager.cc:1939] T 020e8cbbe37540789e5b4dc3b31377e5 P ca961619ac6547939cb7a50a8b68db59: Deleting consensus metadata
I20260925 00:43:03.295969 8142 catalog_manager.cc:4841] TS ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935): tablet 020e8cbbe37540789e5b4dc3b31377e5 (table test-table [id=3631b79cbd7041f58bba32ceaa4ba6a4]) successfully deleted
I20260925 00:43:03.296235 8745 ts_tablet_manager.cc:1905] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:03.299430 8745 ts_tablet_manager.cc:1918] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:03.299521 8745 log.cc:1196] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/020e8cbbe37540789e5b4dc3b31377e5
I20260925 00:43:03.299767 8745 ts_tablet_manager.cc:1939] T 020e8cbbe37540789e5b4dc3b31377e5 P 0d7c45c8384e4b07938ffa7a97f33a0f: Deleting consensus metadata
I20260925 00:43:03.301283 8744 ts_tablet_manager.cc:1939] T 020e8cbbe37540789e5b4dc3b31377e5 P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting consensus metadata
I20260925 00:43:03.301748 8144 catalog_manager.cc:4841] TS 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65:38829): tablet 020e8cbbe37540789e5b4dc3b31377e5 (table test-table [id=3631b79cbd7041f58bba32ceaa4ba6a4]) successfully deleted
I20260925 00:43:03.302081 8142 catalog_manager.cc:4841] TS eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787): tablet 020e8cbbe37540789e5b4dc3b31377e5 (table test-table [id=3631b79cbd7041f58bba32ceaa4ba6a4]) successfully deleted
I20260925 00:43:03.305856 8294 tablet_service.cc:1467] Processing CreateTablet for tablet 4c48e0f4a6cf4a51bdac0b066ca661a3 (DEFAULT_TABLE table=test-table [id=0dd515420d7744a18a942fb5b3e504dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:03.306452 8294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c48e0f4a6cf4a51bdac0b066ca661a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:03.310001 8434 tablet_service.cc:1467] Processing CreateTablet for tablet 4c48e0f4a6cf4a51bdac0b066ca661a3 (DEFAULT_TABLE table=test-table [id=0dd515420d7744a18a942fb5b3e504dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:03.310307 8434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c48e0f4a6cf4a51bdac0b066ca661a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:03.312547 8574 tablet_service.cc:1467] Processing CreateTablet for tablet 4c48e0f4a6cf4a51bdac0b066ca661a3 (DEFAULT_TABLE table=test-table [id=0dd515420d7744a18a942fb5b3e504dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:03.312841 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c48e0f4a6cf4a51bdac0b066ca661a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:03.315135 8756 tablet_bootstrap.cc:492] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac: Bootstrap starting.
I20260925 00:43:03.316010 8755 tablet_bootstrap.cc:492] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59: Bootstrap starting.
I20260925 00:43:03.316052 8756 tablet_bootstrap.cc:654] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:03.317138 8755 tablet_bootstrap.cc:654] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:03.317356 8756 tablet_bootstrap.cc:492] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac: No bootstrap required, opened a new log
I20260925 00:43:03.317448 8756 ts_tablet_manager.cc:1397] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 00:43:03.317838 8756 raft_consensus.cc:348] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.317953 8756 raft_consensus.cc:374] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:03.317988 8756 raft_consensus.cc:729] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf3cf8f75fb41ac8192cc5904cca3ac, State: Initialized, Role: FOLLOWER
I20260925 00:43:03.318107 8757 tablet_bootstrap.cc:492] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f: Bootstrap starting.
I20260925 00:43:03.318104 8756 consensus_queue.cc:260] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.318342 8756 ts_tablet_manager.cc:1428] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:03.319028 8757 tablet_bootstrap.cc:654] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:03.319960 8755 tablet_bootstrap.cc:492] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59: No bootstrap required, opened a new log
I20260925 00:43:03.320048 8755 ts_tablet_manager.cc:1397] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 00:43:03.320711 8757 tablet_bootstrap.cc:492] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f: No bootstrap required, opened a new log
I20260925 00:43:03.320802 8757 ts_tablet_manager.cc:1397] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260925 00:43:03.321264 8757 raft_consensus.cc:348] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.321386 8757 raft_consensus.cc:374] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:03.321429 8757 raft_consensus.cc:729] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d7c45c8384e4b07938ffa7a97f33a0f, State: Initialized, Role: FOLLOWER
I20260925 00:43:03.321563 8757 consensus_queue.cc:260] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.321856 8757 ts_tablet_manager.cc:1428] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 00:43:03.322589 8755 raft_consensus.cc:348] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.322737 8755 raft_consensus.cc:374] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:03.322779 8755 raft_consensus.cc:729] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca961619ac6547939cb7a50a8b68db59, State: Initialized, Role: FOLLOWER
I20260925 00:43:03.322934 8755 consensus_queue.cc:260] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.323273 8755 ts_tablet_manager.cc:1428] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260925 00:43:03.395196 8735 raft_consensus.cc:482] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:03.395375 8735 raft_consensus.cc:504] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.395884 8735 leader_election.cc:288] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935), eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787)
I20260925 00:43:03.396520 8454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c48e0f4a6cf4a51bdac0b066ca661a3" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca961619ac6547939cb7a50a8b68db59" is_pre_election: true
I20260925 00:43:03.396552 8594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c48e0f4a6cf4a51bdac0b066ca661a3" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" is_pre_election: true
I20260925 00:43:03.396688 8454 raft_consensus.cc:2454] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 0.
I20260925 00:43:03.396688 8594 raft_consensus.cc:2454] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 0.
I20260925 00:43:03.397030 8246 leader_election.cc:302] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d7c45c8384e4b07938ffa7a97f33a0f, eaf3cf8f75fb41ac8192cc5904cca3ac; no voters:
I20260925 00:43:03.397253 8735 raft_consensus.cc:2781] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:03.397315 8735 raft_consensus.cc:482] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:03.397367 8735 raft_consensus.cc:3037] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:03.398535 8735 raft_consensus.cc:504] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.398954 8735 leader_election.cc:288] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 election: Requested vote from peers ca961619ac6547939cb7a50a8b68db59 (127.6.72.66:32935), eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787)
I20260925 00:43:03.399411 8454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c48e0f4a6cf4a51bdac0b066ca661a3" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca961619ac6547939cb7a50a8b68db59"
I20260925 00:43:03.399415 8594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c48e0f4a6cf4a51bdac0b066ca661a3" candidate_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac"
I20260925 00:43:03.399549 8594 raft_consensus.cc:3037] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:03.399554 8454 raft_consensus.cc:3037] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:03.400691 8454 raft_consensus.cc:2454] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 1.
I20260925 00:43:03.400825 8594 raft_consensus.cc:2454] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d7c45c8384e4b07938ffa7a97f33a0f in term 1.
I20260925 00:43:03.401242 8246 leader_election.cc:302] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d7c45c8384e4b07938ffa7a97f33a0f, eaf3cf8f75fb41ac8192cc5904cca3ac; no voters:
I20260925 00:43:03.401450 8735 raft_consensus.cc:2781] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:03.401578 8735 raft_consensus.cc:686] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [term 1 LEADER]: Becoming Leader. State: Replica: 0d7c45c8384e4b07938ffa7a97f33a0f, State: Running, Role: LEADER
I20260925 00:43:03.401736 8735 consensus_queue.cc:237] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } }
I20260925 00:43:03.403779 8156 catalog_manager.cc:5495] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65). New cstate: current_term: 1 leader_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d7c45c8384e4b07938ffa7a97f33a0f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 } health_report { overall_health: UNKNOWN } } }
I20260925 00:43:03.423563 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:03.419271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.6.72.65:38829)
0925 00:43:03.419678 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:03.419688 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:03.419703 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:43:03.419750 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:03.419754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:03.419803 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:03.419887 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:03.420179 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.420755 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.422325 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:03.422537 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:03.422625 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:03.422958 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:03.423036 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:03.423114 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:03.423279 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 00:43:03.423333 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":122,"threads_started":1}
I20260925 00:43:03.431097 8454 raft_consensus.cc:1261] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Refusing update from remote peer 0d7c45c8384e4b07938ffa7a97f33a0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:03.431095 8594 raft_consensus.cc:1261] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Refusing update from remote peer 0d7c45c8384e4b07938ffa7a97f33a0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:03.431579 8735 consensus_queue.cc:1035] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf3cf8f75fb41ac8192cc5904cca3ac" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:03.431740 8674 consensus_queue.cc:1035] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P 0d7c45c8384e4b07938ffa7a97f33a0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca961619ac6547939cb7a50a8b68db59" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 32935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 00:43:03.448277 8727 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 00:43:05.006383 8157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:05.006562 8157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:05.009341 8157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 443b32f18e8e43de8944f891092fc2a1: Sending DeleteTablet for 3 replicas of tablet 4c48e0f4a6cf4a51bdac0b066ca661a3
I20260925 00:43:05.010136 8434 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c48e0f4a6cf4a51bdac0b066ca661a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:05 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:32966
I20260925 00:43:05.010205 8294 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c48e0f4a6cf4a51bdac0b066ca661a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:05 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:44226
I20260925 00:43:05.010466 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 8229
I20260925 00:43:05.010591 8574 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c48e0f4a6cf4a51bdac0b066ca661a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:05 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:56824
I20260925 00:43:05.012522 8808 tablet_replica.cc:330] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac: stopping tablet replica
I20260925 00:43:05.012717 8808 raft_consensus.cc:2229] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:05.012877 8808 raft_consensus.cc:2258] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:05.013636 8808 ts_tablet_manager.cc:1905] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:05.017194 8808 ts_tablet_manager.cc:1918] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 00:43:05.017290 8808 log.cc:1196] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/4c48e0f4a6cf4a51bdac0b066ca661a3
I20260925 00:43:05.017565 8808 ts_tablet_manager.cc:1939] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P eaf3cf8f75fb41ac8192cc5904cca3ac: Deleting consensus metadata
I20260925 00:43:05.017556 8806 tablet_replica.cc:330] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59: stopping tablet replica
I20260925 00:43:05.017750 8806 raft_consensus.cc:2229] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:05.017915 8806 raft_consensus.cc:2258] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:05.018482 8142 catalog_manager.cc:4841] TS eaf3cf8f75fb41ac8192cc5904cca3ac (127.6.72.67:45787): tablet 4c48e0f4a6cf4a51bdac0b066ca661a3 (table test-table [id=0dd515420d7744a18a942fb5b3e504dd]) successfully deleted
I20260925 00:43:05.018618 8806 ts_tablet_manager.cc:1905] T 4c48e0f4a6cf4a51bdac0b066ca661a3 P ca961619ac6547939cb7a50a8b68db59: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 00:43:05.020113 8144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:38829 (error 108)
I20260925 00:43:05.020408 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 8369
I20260925 00:43:05.021750 8810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:05.021259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:38829 (local address 127.0.0.1:44248)
0925 00:43:05.021533 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:05.021621 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:38829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
W20260925 00:43:05.021906 8144 catalog_manager.cc:4584] TS 0d7c45c8384e4b07938ffa7a97f33a0f (127.6.72.65:38829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c48e0f4a6cf4a51bdac0b066ca661a3: Network error: Client connection negotiation failed: client connection to 127.6.72.65:38829: connect: Connection refused (error 111)
I20260925 00:43:05.028295 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 8509
I20260925 00:43:05.029450 8810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:05.029098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:32935 (local address 127.0.0.1:32988)
0925 00:43:05.029291 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:05.029351 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.66:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260925 00:43:05.034740 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 8125
2026-09-25T00:43:05Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6413 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:05 dist-test-slave-w8nh krb5kdc[8813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:05 dist-test-slave-w8nh krb5kdc[8813](info): set up 2 sockets
Sep 25 00:43:05 dist-test-slave-w8nh krb5kdc[8813](info): commencing operation
krb5kdc: starting...
W20260925 00:43:06.101468 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:06 dist-test-slave-w8nh krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
W20260925 00:43:07.301357 6433 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.6.72.126: real 1.096s user 0.002s sys 0.002s
I20260925 00:43:07.302151 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:40039
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:40039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:07.424638 8826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:07.424914 8826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:07.424957 8826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:07.429085 8826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:07.429140 8826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:07.429160 8826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:07.429179 8826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:07.429234 8826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:07.434504 8826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:40039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:40039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:07.435616 8826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:07.436771 8826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 00:43:07.445713 8826 server_base.cc:1034] running on GCE node
W20260925 00:43:07.445600 8831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:07.445639 8832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:07.445638 8834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:07.447691 8826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:07.448896 8826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:07.450220 8826 hybrid_clock.cc:648] HybridClock initialized: now 1790296987450222 us; error 63 us; skew 500 ppm
Sep 25 00:43:07 dist-test-slave-w8nh krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296987, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:07.457607 8826 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:43:07.458820 8826 webserver.cc:460] Webserver started at http://127.6.72.126:34421/ using document root <none> and password file <none>
I20260925 00:43:07.459475 8826 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:07.459574 8826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:07.460212 8826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:07.463665 8826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "76d40d291ddf404ea40a9c92142b19f4"
format_stamp: "Formatted at 2026-09-25 00:43:07 on dist-test-slave-w8nh"
server_key: "4fdde1dcd6171d09dd1a57b80a2a1a4c"
server_key_iv: "404447a829e5a9bda85431060d588f08"
server_key_version: "encryptionkey@0"
I20260925 00:43:07.464478 8826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "76d40d291ddf404ea40a9c92142b19f4"
format_stamp: "Formatted at 2026-09-25 00:43:07 on dist-test-slave-w8nh"
server_key: "4fdde1dcd6171d09dd1a57b80a2a1a4c"
server_key_iv: "404447a829e5a9bda85431060d588f08"
server_key_version: "encryptionkey@0"
I20260925 00:43:07.468874 8826 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 00:43:07.472329 8841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:07.474489 8826 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:43:07.474713 8826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "76d40d291ddf404ea40a9c92142b19f4"
format_stamp: "Formatted at 2026-09-25 00:43:07 on dist-test-slave-w8nh"
server_key: "4fdde1dcd6171d09dd1a57b80a2a1a4c"
server_key_iv: "404447a829e5a9bda85431060d588f08"
server_key_version: "encryptionkey@0"
I20260925 00:43:07.474871 8826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:07.493729 8826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:07.497611 8826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:07.497844 8826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:07.506094 8826 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:40039
I20260925 00:43:07.506111 8893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:40039 every 8 connection(s)
I20260925 00:43:07.507174 8826 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:07.510159 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 8826
I20260925 00:43:07.510284 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:07.510448 8894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:07.510581 6433 external_mini_cluster.cc:1428] Setting key 65f7cbf6fc3d3723f7307d9220003066
I20260925 00:43:07.517251 8894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4: Bootstrap starting.
Sep 25 00:43:07 dist-test-slave-w8nh krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:07.519588 8894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:07.520506 8894 log.cc:824] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:07.522421 8894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4: No bootstrap required, opened a new log
I20260925 00:43:07.524892 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:07.512269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.6.72.126:40039)
0925 00:43:07.512782 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:07.512799 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:07.512840 (+ 41us) server_negotiation.cc:407] Connection header received
0925 00:43:07.513503 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:07.513521 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:07.513885 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:07.514306 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:07.515077 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:07.516082 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:07.516995 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:07.517537 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:07.517706 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:07.520152 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:07.520158 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:07.520174 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:07.520205 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:07.522033 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:07.522484 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:07.522490 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:07.522494 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:07.522648 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:07.522936 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:07.522941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:07.522942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:07.523327 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:07.523521 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:07.523828 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 00:43:07.524103 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":134,"threads_started":1}
I20260925 00:43:07.526186 8894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d40d291ddf404ea40a9c92142b19f4" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40039 } }
I20260925 00:43:07.526424 8894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:07.526515 8894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d40d291ddf404ea40a9c92142b19f4, State: Initialized, Role: FOLLOWER
I20260925 00:43:07.527009 8894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d40d291ddf404ea40a9c92142b19f4" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40039 } }
I20260925 00:43:07.527170 8894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:07.527256 8894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:07.527349 8894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:07.579747 8894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d40d291ddf404ea40a9c92142b19f4" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40039 } }
I20260925 00:43:07.580498 8894 leader_election.cc:302] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76d40d291ddf404ea40a9c92142b19f4; no voters:
I20260925 00:43:07.580971 8894 leader_election.cc:288] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:07.581177 8899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:07.581439 8899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [term 1 LEADER]: Becoming Leader. State: Replica: 76d40d291ddf404ea40a9c92142b19f4, State: Running, Role: LEADER
I20260925 00:43:07.581871 8899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d40d291ddf404ea40a9c92142b19f4" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40039 } }
I20260925 00:43:07.582616 8894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:07.583709 8901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76d40d291ddf404ea40a9c92142b19f4. Latest consensus state: current_term: 1 leader_uuid: "76d40d291ddf404ea40a9c92142b19f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d40d291ddf404ea40a9c92142b19f4" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40039 } } }
I20260925 00:43:07.583858 8901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:07.584015 8900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76d40d291ddf404ea40a9c92142b19f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d40d291ddf404ea40a9c92142b19f4" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40039 } } }
I20260925 00:43:07.584120 8900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:07.584457 8908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:07.588201 8908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:07.594863 8908 catalog_manager.cc:1347] Generated new cluster ID: 00ac5decb7e84775bd70d36cbf21a9cf
I20260925 00:43:07.594986 8908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:07.606591 8908 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:07.607659 8908 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:07.617031 8908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76d40d291ddf404ea40a9c92142b19f4: Generated new TSK 0
I20260925 00:43:07.617853 8908 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
W20260925 00:43:09.038236 6433 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.6.72.65: real 1.416s user 0.002s sys 0.002s
I20260925 00:43:09.038770 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40039
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:09.169461 8920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:09.169833 8920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:09.169920 8920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:09.174302 8920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:09.174419 8920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:09.174520 8920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:09.180528 8920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:09.181841 8920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:09.183064 8920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:09.194317 8925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:09.194339 8928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:09.194357 8926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:09.194726 8920 server_base.cc:1034] running on GCE node
I20260925 00:43:09.195215 8920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:09.196165 8920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:09.197435 8920 hybrid_clock.cc:648] HybridClock initialized: now 1790296989197395 us; error 58 us; skew 500 ppm
Sep 25 00:43:09 dist-test-slave-w8nh krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296989, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:09.201496 8920 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:43:09.202652 8920 webserver.cc:460] Webserver started at http://127.6.72.65:40055/ using document root <none> and password file <none>
I20260925 00:43:09.203397 8920 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:09.203511 8920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:09.203794 8920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:09.206396 8920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "34bc51855b1647bc9ce03666f43869d9"
format_stamp: "Formatted at 2026-09-25 00:43:09 on dist-test-slave-w8nh"
server_key: "5aa49b3b443e40f059961834729e3e6a"
server_key_iv: "a5f2d1acaba44c7a11a44a9c8cb89cbd"
server_key_version: "encryptionkey@0"
I20260925 00:43:09.207154 8920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "34bc51855b1647bc9ce03666f43869d9"
format_stamp: "Formatted at 2026-09-25 00:43:09 on dist-test-slave-w8nh"
server_key: "5aa49b3b443e40f059961834729e3e6a"
server_key_iv: "a5f2d1acaba44c7a11a44a9c8cb89cbd"
server_key_version: "encryptionkey@0"
I20260925 00:43:09.211954 8920 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 00:43:09.215013 8935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:09.216648 8920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:09.216816 8920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "34bc51855b1647bc9ce03666f43869d9"
format_stamp: "Formatted at 2026-09-25 00:43:09 on dist-test-slave-w8nh"
server_key: "5aa49b3b443e40f059961834729e3e6a"
server_key_iv: "a5f2d1acaba44c7a11a44a9c8cb89cbd"
server_key_version: "encryptionkey@0"
I20260925 00:43:09.216960 8920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:09.239611 8920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:09.242972 8920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:09.243201 8920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:09.243835 8920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:09.244808 8920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:09.244879 8920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:09.244956 8920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:09.245011 8920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:09.255771 8920 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:33437
I20260925 00:43:09.255795 9048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:33437 every 8 connection(s)
I20260925 00:43:09.256901 8920 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:09.257900 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 8920
I20260925 00:43:09.258038 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:09.258318 6433 external_mini_cluster.cc:1428] Setting key 708eb1116e146ada73bc321e58b41440
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
Sep 25 00:43:09 dist-test-slave-w8nh krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296989, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:09.278191 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:09.259450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:57559 (local address 127.6.72.126:40039)
0925 00:43:09.259765 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:09.259785 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:09.260899 (+ 1114us) server_negotiation.cc:407] Connection header received
0925 00:43:09.263416 (+ 2517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:09.263420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:09.263484 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:09.263582 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:09.265767 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:09.266521 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:09.268827 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:09.269245 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:09.269404 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:09.273419 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:09.273423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:09.273425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:09.273455 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:09.275535 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:09.276323 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:09.276333 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:09.276335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:09.276430 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:09.276897 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:09.276900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:09.276902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:09.277167 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:09.277256 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:09.277893 (+ 637us) server_negotiation.cc:299] Negotiation successful
0925 00:43:09.278022 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":124,"threads_started":1}
I20260925 00:43:09.279155 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:09.260086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:40039 (local address 127.6.72.65:57559)
0925 00:43:09.260706 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:09.260770 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:09.262953 (+ 2183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:09.263851 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:09.263865 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:09.264546 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:09.265442 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:09.265472 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:09.266772 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:09.266779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:09.268586 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:09.268604 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:09.269701 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:09.269717 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:09.270001 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:09.270766 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:09.270805 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:09.273098 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:09.275709 (+ 2611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:09.275716 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:09.275734 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:09.276089 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:09.276549 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:09.276552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:09.276554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:09.276742 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:09.277289 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:09.277295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:09.277608 (+ 313us) client_negotiation.cc:769] Sending connection context
0925 00:43:09.277790 (+ 182us) client_negotiation.cc:240] Negotiation successful
0925 00:43:09.278096 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":196,"threads_started":1}
I20260925 00:43:09.280959 9049 heartbeater.cc:347] Connected to a master server at 127.6.72.126:40039
I20260925 00:43:09.281320 9049 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:09.282012 9049 heartbeater.cc:511] Master 127.6.72.126:40039 requested a full tablet report, sending...
I20260925 00:43:09.284001 8858 ts_manager.cc:194] Registered new tserver with Master: 34bc51855b1647bc9ce03666f43869d9 (127.6.72.65:33437)
I20260925 00:43:09.285471 8858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:57559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
W20260925 00:43:10.191354 6433 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.6.72.66: real 0.931s user 0.001s sys 0.002s
I20260925 00:43:10.191905 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40039
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:43:10.288300 9049 heartbeater.cc:503] Master 127.6.72.126:40039 was elected leader, sending a full tablet report...
W20260925 00:43:10.312139 9055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:10.312506 9055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:10.312579 9055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:10.316805 9055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:10.316874 9055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:10.316937 9055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:10.322567 9055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:10.323681 9055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:10.324759 9055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:10.334856 9060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:10.334839 9063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:10.334839 9061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:10.335206 9055 server_base.cc:1034] running on GCE node
I20260925 00:43:10.335709 9055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:10.336468 9055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:10.337689 9055 hybrid_clock.cc:648] HybridClock initialized: now 1790296990337674 us; error 27 us; skew 500 ppm
Sep 25 00:43:10 dist-test-slave-w8nh krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296990, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:10.340916 9055 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:43:10.341856 9055 webserver.cc:460] Webserver started at http://127.6.72.66:43261/ using document root <none> and password file <none>
I20260925 00:43:10.342442 9055 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:10.342504 9055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:10.342753 9055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:10.344972 9055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "fafe32ef7edf4cc8abbe187edb1a069a"
format_stamp: "Formatted at 2026-09-25 00:43:10 on dist-test-slave-w8nh"
server_key: "dc46b75a877a95deba33b8dbc71bfa06"
server_key_iv: "b54307c04c677b0c8941717ec18ff7cd"
server_key_version: "encryptionkey@0"
I20260925 00:43:10.345574 9055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "fafe32ef7edf4cc8abbe187edb1a069a"
format_stamp: "Formatted at 2026-09-25 00:43:10 on dist-test-slave-w8nh"
server_key: "dc46b75a877a95deba33b8dbc71bfa06"
server_key_iv: "b54307c04c677b0c8941717ec18ff7cd"
server_key_version: "encryptionkey@0"
I20260925 00:43:10.349566 9055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:43:10.352123 9070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:10.353497 9055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:10.353639 9055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "fafe32ef7edf4cc8abbe187edb1a069a"
format_stamp: "Formatted at 2026-09-25 00:43:10 on dist-test-slave-w8nh"
server_key: "dc46b75a877a95deba33b8dbc71bfa06"
server_key_iv: "b54307c04c677b0c8941717ec18ff7cd"
server_key_version: "encryptionkey@0"
I20260925 00:43:10.353760 9055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:10.370640 9055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:10.374329 9055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:10.374565 9055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:10.375244 9055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:10.376139 9055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:10.376214 9055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:10.376304 9055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:10.376353 9055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:10.386148 9055 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:36853
I20260925 00:43:10.386166 9183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:36853 every 8 connection(s)
I20260925 00:43:10.387130 9055 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:10.388949 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 9055
I20260925 00:43:10.389087 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:10.389417 6433 external_mini_cluster.cc:1428] Setting key f66c9d70ad50bff4901992f1ed31d02c
Sep 25 00:43:10 dist-test-slave-w8nh krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296990, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:43:10.404294 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:10.389067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:40873 (local address 127.6.72.126:40039)
0925 00:43:10.389365 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:10.389371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:10.390148 (+ 777us) server_negotiation.cc:407] Connection header received
0925 00:43:10.392197 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:10.392210 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:10.392298 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:10.392409 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:10.394135 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:10.394695 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:10.396660 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:10.396906 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:10.397011 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:10.400113 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:10.400117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:10.400119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:10.400144 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:10.401781 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:10.402320 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:10.402326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:10.402328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:10.402414 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:10.402869 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:10.402873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:10.402875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:10.403141 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:10.403293 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:10.403827 (+ 534us) server_negotiation.cc:299] Negotiation successful
0925 00:43:10.403984 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":89,"threads_started":1}
I20260925 00:43:10.405352 9187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:10.389476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:40039 (local address 127.6.72.66:40873)
0925 00:43:10.389981 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:10.390039 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:10.391964 (+ 1925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:10.392653 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:10.392664 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:10.393224 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:10.393956 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:10.393979 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:10.394840 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:10.394846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:10.396496 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:10.396512 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:10.397092 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:10.397098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:10.397342 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:10.398076 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:10.398101 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:10.399904 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:10.401909 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:10.401915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:10.401932 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:10.402212 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:10.402537 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:10.402544 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:10.402549 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:10.402765 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:10.403308 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:10.403314 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:10.403574 (+ 260us) client_negotiation.cc:769] Sending connection context
0925 00:43:10.403755 (+ 181us) client_negotiation.cc:240] Negotiation successful
0925 00:43:10.404038 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":124,"threads_started":1}
I20260925 00:43:10.406749 9184 heartbeater.cc:347] Connected to a master server at 127.6.72.126:40039
I20260925 00:43:10.407035 9184 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:10.407569 9184 heartbeater.cc:511] Master 127.6.72.126:40039 requested a full tablet report, sending...
I20260925 00:43:10.408792 8858 ts_manager.cc:194] Registered new tserver with Master: fafe32ef7edf4cc8abbe187edb1a069a (127.6.72.66:36853)
I20260925 00:43:10.409648 8858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:40873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:43:10.419955 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40039
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:10.552127 9190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:10.552541 9190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:10.552589 9190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:10.557006 9190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:10.557070 9190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:10.557132 9190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:10.562829 9190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:10.563913 9190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:10.565006 9190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:10.574915 9195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:10.574915 9196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:10.575022 9198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:10.575300 9190 server_base.cc:1034] running on GCE node
I20260925 00:43:10.575855 9190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:10.576609 9190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:10.577803 9190 hybrid_clock.cc:648] HybridClock initialized: now 1790296990577781 us; error 35 us; skew 500 ppm
Sep 25 00:43:10 dist-test-slave-w8nh krb5kdc[8813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296990, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:10.581086 9190 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:43:10.582063 9190 webserver.cc:460] Webserver started at http://127.6.72.67:45817/ using document root <none> and password file <none>
I20260925 00:43:10.582757 9190 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:10.582851 9190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:10.583097 9190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:10.585619 9190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "ace16beceae64ddd9fdebe51c77f0f54"
format_stamp: "Formatted at 2026-09-25 00:43:10 on dist-test-slave-w8nh"
server_key: "4b7ce4ae94e00e5a68998ee181123a96"
server_key_iv: "ab1d28772089307e072b6fb37f43cba1"
server_key_version: "encryptionkey@0"
I20260925 00:43:10.586277 9190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "ace16beceae64ddd9fdebe51c77f0f54"
format_stamp: "Formatted at 2026-09-25 00:43:10 on dist-test-slave-w8nh"
server_key: "4b7ce4ae94e00e5a68998ee181123a96"
server_key_iv: "ab1d28772089307e072b6fb37f43cba1"
server_key_version: "encryptionkey@0"
I20260925 00:43:10.590564 9190 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:43:10.594120 9205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:10.595608 9190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 00:43:10.595788 9190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "ace16beceae64ddd9fdebe51c77f0f54"
format_stamp: "Formatted at 2026-09-25 00:43:10 on dist-test-slave-w8nh"
server_key: "4b7ce4ae94e00e5a68998ee181123a96"
server_key_iv: "ab1d28772089307e072b6fb37f43cba1"
server_key_version: "encryptionkey@0"
I20260925 00:43:10.595927 9190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:10.637938 9190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:10.641542 9190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:10.641769 9190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:10.642406 9190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:10.643270 9190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:10.643343 9190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:10.643416 9190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:10.643469 9190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:10.652906 9190 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:34693
I20260925 00:43:10.652925 9318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:34693 every 8 connection(s)
I20260925 00:43:10.653858 9190 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:10.658053 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 9190
I20260925 00:43:10.658185 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:10.658494 6433 external_mini_cluster.cc:1428] Setting key 6156ce84beca247042b3a4cbab3810bc
Sep 25 00:43:10 dist-test-slave-w8nh krb5kdc[8813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296990, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:10.668725 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:10.655787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:49779 (local address 127.6.72.126:40039)
0925 00:43:10.656413 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:10.656420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:10.657121 (+ 701us) server_negotiation.cc:407] Connection header received
0925 00:43:10.658670 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:10.658674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:10.658732 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:10.658841 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:10.660403 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:10.660958 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:10.662148 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:10.662389 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:10.662503 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:10.665170 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:10.665173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:10.665175 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:10.665205 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:10.666492 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:10.667070 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:10.667074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:10.667075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:10.667152 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:10.667522 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:10.667525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:10.667527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:10.667723 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:10.667833 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:10.668410 (+ 577us) server_negotiation.cc:299] Negotiation successful
0925 00:43:10.668552 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518}
I20260925 00:43:10.669587 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:10.656358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:40039 (local address 127.6.72.67:49779)
0925 00:43:10.656920 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:10.656972 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:10.658447 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:10.659091 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:10.659101 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:10.659607 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:10.660212 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:10.660226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:10.661112 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:10.661116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:10.662012 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:10.662021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:10.662547 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:10.662551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:10.662723 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:10.663388 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:10.663405 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:10.664968 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:10.666628 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:10.666635 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:10.666653 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:10.666947 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:10.667255 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:10.667258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:10.667260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:10.667414 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:10.667843 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:10.667852 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:10.668121 (+ 269us) client_negotiation.cc:769] Sending connection context
0925 00:43:10.668350 (+ 229us) client_negotiation.cc:240] Negotiation successful
0925 00:43:10.668715 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":132,"threads_started":1}
I20260925 00:43:10.670725 9319 heartbeater.cc:347] Connected to a master server at 127.6.72.126:40039
I20260925 00:43:10.670957 9319 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:10.671563 9319 heartbeater.cc:511] Master 127.6.72.126:40039 requested a full tablet report, sending...
I20260925 00:43:10.672752 8858 ts_manager.cc:194] Registered new tserver with Master: ace16beceae64ddd9fdebe51c77f0f54 (127.6.72.67:34693)
I20260925 00:43:10.673288 8858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:49779
I20260925 00:43:10.685611 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 00:43:10.699082 9186 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:10.697630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49722 (local address 127.6.72.126:40039)
0925 00:43:10.697804 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:10.697812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:10.697944 (+ 132us) server_negotiation.cc:407] Connection header received
0925 00:43:10.698035 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:10.698038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:10.698092 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:10.698167 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:10.698927 (+ 760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49722: BlockingRecv error: recv got EOF from 127.0.0.1:49722 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 00:43:10.700466 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 8920
I20260925 00:43:10.706094 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 9055
I20260925 00:43:10.711442 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 9190
I20260925 00:43:10.716770 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 8826
2026-09-25T00:43:10Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (5679 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:10 dist-test-slave-w8nh krb5kdc[9331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:10 dist-test-slave-w8nh krb5kdc[9331](info): set up 2 sockets
Sep 25 00:43:10 dist-test-slave-w8nh krb5kdc[9331](info): commencing operation
krb5kdc: starting...
W20260925 00:43:11.788780 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:11 dist-test-slave-w8nh krb5kdc[9331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:43:11.916412 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:37689
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:33827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:37689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:12.030905 9344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:12.031265 9344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:12.031339 9344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:12.035626 9344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:12.035742 9344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:12.035780 9344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:12.035820 9344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:12.035861 9344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:12.041603 9344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:33827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:37689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:37689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:12.042799 9344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:12.043917 9344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:12.052963 9350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:12.052989 9349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:12.053062 9344 server_base.cc:1034] running on GCE node
W20260925 00:43:12.052978 9352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:12.053959 9344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:12.055068 9344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:12.056348 9344 hybrid_clock.cc:648] HybridClock initialized: now 1790296992056313 us; error 53 us; skew 500 ppm
Sep 25 00:43:12 dist-test-slave-w8nh krb5kdc[9331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296992, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:12.059561 9344 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:43:12.060714 9344 webserver.cc:460] Webserver started at http://127.6.72.126:46769/ using document root <none> and password file <none>
I20260925 00:43:12.061333 9344 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:12.061432 9344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:12.061678 9344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:12.063866 9344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "5f95f50faad84f6caf7135a83ae66560"
format_stamp: "Formatted at 2026-09-25 00:43:12 on dist-test-slave-w8nh"
server_key: "0b0072d8923a09d5d958ff21daaf45ae"
server_key_iv: "976ea70b82bc3a5e3058c67e60f43464"
server_key_version: "encryptionkey@0"
I20260925 00:43:12.064496 9344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "5f95f50faad84f6caf7135a83ae66560"
format_stamp: "Formatted at 2026-09-25 00:43:12 on dist-test-slave-w8nh"
server_key: "0b0072d8923a09d5d958ff21daaf45ae"
server_key_iv: "976ea70b82bc3a5e3058c67e60f43464"
server_key_version: "encryptionkey@0"
I20260925 00:43:12.068404 9344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:43:12.070869 9359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:12.072122 9344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:43:12.072331 9344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "5f95f50faad84f6caf7135a83ae66560"
format_stamp: "Formatted at 2026-09-25 00:43:12 on dist-test-slave-w8nh"
server_key: "0b0072d8923a09d5d958ff21daaf45ae"
server_key_iv: "976ea70b82bc3a5e3058c67e60f43464"
server_key_version: "encryptionkey@0"
I20260925 00:43:12.072464 9344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:12.082461 9344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:12.085455 9344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:12.085667 9344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:12.092928 9344 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:37689
I20260925 00:43:12.093272 9411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:37689 every 8 connection(s)
I20260925 00:43:12.093920 9344 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:12.096894 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:12.102414 9412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560: Bootstrap starting.
I20260925 00:43:12.103003 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 9344
I20260925 00:43:12.103111 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:12.103387 6433 external_mini_cluster.cc:1428] Setting key 212a58f2b81023fff372d50bf0856f84
I20260925 00:43:12.105242 9412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:12.105983 9412 log.cc:824] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:12.108135 9412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560: No bootstrap required, opened a new log
Sep 25 00:43:12 dist-test-slave-w8nh krb5kdc[9331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:12.112792 9412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f95f50faad84f6caf7135a83ae66560" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37689 } }
I20260925 00:43:12.113054 9412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:12.113153 9412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f95f50faad84f6caf7135a83ae66560, State: Initialized, Role: FOLLOWER
I20260925 00:43:12.113709 9412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f95f50faad84f6caf7135a83ae66560" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37689 } }
I20260925 00:43:12.113895 9412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:12.113976 9412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:12.114092 9412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:12.115034 9412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f95f50faad84f6caf7135a83ae66560" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37689 } }
I20260925 00:43:12.115408 9412 leader_election.cc:302] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f95f50faad84f6caf7135a83ae66560; no voters:
I20260925 00:43:12.115761 9412 leader_election.cc:288] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:12.115841 9417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:12.116053 9417 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [term 1 LEADER]: Becoming Leader. State: Replica: 5f95f50faad84f6caf7135a83ae66560, State: Running, Role: LEADER
I20260925 00:43:12.116434 9417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f95f50faad84f6caf7135a83ae66560" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37689 } }
I20260925 00:43:12.117131 9412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:12.117671 9417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f95f50faad84f6caf7135a83ae66560. Latest consensus state: current_term: 1 leader_uuid: "5f95f50faad84f6caf7135a83ae66560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f95f50faad84f6caf7135a83ae66560" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37689 } } }
I20260925 00:43:12.117882 9417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:12.118098 9418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f95f50faad84f6caf7135a83ae66560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f95f50faad84f6caf7135a83ae66560" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 37689 } } }
I20260925 00:43:12.118258 9418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:12.118438 9415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:12.104715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59738 (local address 127.6.72.126:37689)
0925 00:43:12.105159 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:12.105173 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:12.105207 (+ 34us) server_negotiation.cc:407] Connection header received
0925 00:43:12.105881 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:12.105898 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:12.106280 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:12.106670 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:12.107538 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:12.108603 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:12.109742 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:12.110191 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:12.110302 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:12.113215 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:12.113222 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:12.113242 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:12.113274 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:12.115014 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:12.115489 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:12.115495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:12.115508 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:12.115663 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:12.115982 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:12.115985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:12.115987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:12.116504 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:12.116763 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:12.117044 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 00:43:12.117385 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":138,"threads_started":1}
I20260925 00:43:12.121361 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260925 00:43:12.122363 9432 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:43:12.122434 9432 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:43:12.122543 9424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:12.123431 9424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:12.129215 9424 catalog_manager.cc:1347] Generated new cluster ID: 25d731c6ec28432581d5a1f504960ec8
I20260925 00:43:12.129366 9424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:12.142333 9415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:12.136543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.6.72.126:37689)
0925 00:43:12.136660 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:12.136665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:12.136996 (+ 331us) server_negotiation.cc:407] Connection header received
0925 00:43:12.137107 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:12.137110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:12.137166 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:12.137253 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:12.137784 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:12.138583 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:12.139713 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:12.140098 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:12.140202 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:12.141155 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:12.141159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:12.141162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:12.141765 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:12.141806 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:12.141826 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:12.141919 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:12.142081 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 00:43:12.142148 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 00:43:12.144845 9424 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:12.146008 9424 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:12.157450 9424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f95f50faad84f6caf7135a83ae66560: Generated new TSK 0
I20260925 00:43:12.158339 9424 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:43:12.165223 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 9344
2026-09-25T00:43:12Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1445 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:12 dist-test-slave-w8nh krb5kdc[9445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:12 dist-test-slave-w8nh krb5kdc[9445](info): set up 2 sockets
Sep 25 00:43:12 dist-test-slave-w8nh krb5kdc[9445](info): commencing operation
krb5kdc: starting...
W20260925 00:43:13.212823 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:13 dist-test-slave-w8nh krb5kdc[9445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:43:13.340220 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:36295
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:36295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:13.451143 9458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:13.451488 9458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:13.451562 9458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:13.455691 9458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:13.455806 9458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:13.455843 9458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:13.455900 9458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:13.455956 9458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:13.461490 9458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:36295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:36295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:13.462651 9458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:13.463860 9458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:13.472558 9463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:13.472558 9466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:13.472903 9458 server_base.cc:1034] running on GCE node
W20260925 00:43:13.472608 9464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:13.473472 9458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:13.474612 9458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:13.475804 9458 hybrid_clock.cc:648] HybridClock initialized: now 1790296993475776 us; error 41 us; skew 500 ppm
Sep 25 00:43:13 dist-test-slave-w8nh krb5kdc[9445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:13.484367 9458 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:43:13.485471 9458 webserver.cc:460] Webserver started at http://127.6.72.126:42599/ using document root <none> and password file <none>
I20260925 00:43:13.486090 9458 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:13.486155 9458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:13.486373 9458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:13.488615 9458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "b30027368aba47a2954a87779a49d2a5"
format_stamp: "Formatted at 2026-09-25 00:43:13 on dist-test-slave-w8nh"
server_key: "668693d7474ba021b489d12b7092f508"
server_key_iv: "36c2335e0fc19a60f72cd03f24e9718f"
server_key_version: "encryptionkey@0"
I20260925 00:43:13.489209 9458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "b30027368aba47a2954a87779a49d2a5"
format_stamp: "Formatted at 2026-09-25 00:43:13 on dist-test-slave-w8nh"
server_key: "668693d7474ba021b489d12b7092f508"
server_key_iv: "36c2335e0fc19a60f72cd03f24e9718f"
server_key_version: "encryptionkey@0"
I20260925 00:43:13.493172 9458 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 00:43:13.495764 9473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:13.497072 9458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 00:43:13.497219 9458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "b30027368aba47a2954a87779a49d2a5"
format_stamp: "Formatted at 2026-09-25 00:43:13 on dist-test-slave-w8nh"
server_key: "668693d7474ba021b489d12b7092f508"
server_key_iv: "36c2335e0fc19a60f72cd03f24e9718f"
server_key_version: "encryptionkey@0"
I20260925 00:43:13.497344 9458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:13.519114 9458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:13.522473 9458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:13.522703 9458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:13.529982 9458 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:36295
I20260925 00:43:13.530339 9525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:36295 every 8 connection(s)
I20260925 00:43:13.531039 9458 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:13.534050 9526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:13.537176 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 9458
I20260925 00:43:13.537345 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:13.537618 6433 external_mini_cluster.cc:1428] Setting key 4cacb9fd6d618a0b9ea3fb015ab8df22
I20260925 00:43:13.539963 9526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5: Bootstrap starting.
I20260925 00:43:13.542649 9526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:13.543548 9526 log.cc:824] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:13.545524 9526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5: No bootstrap required, opened a new log
Sep 25 00:43:13 dist-test-slave-w8nh krb5kdc[9445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:13.548650 9526 raft_consensus.cc:348] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30027368aba47a2954a87779a49d2a5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36295 } }
I20260925 00:43:13.548949 9526 raft_consensus.cc:374] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:13.549044 9526 raft_consensus.cc:729] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b30027368aba47a2954a87779a49d2a5, State: Initialized, Role: FOLLOWER
I20260925 00:43:13.549503 9526 consensus_queue.cc:260] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30027368aba47a2954a87779a49d2a5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36295 } }
I20260925 00:43:13.549661 9526 raft_consensus.cc:388] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:13.549748 9526 raft_consensus.cc:482] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:13.549872 9526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:13.550782 9526 raft_consensus.cc:504] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30027368aba47a2954a87779a49d2a5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36295 } }
I20260925 00:43:13.551136 9526 leader_election.cc:302] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b30027368aba47a2954a87779a49d2a5; no voters:
I20260925 00:43:13.551471 9526 leader_election.cc:288] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:13.551707 9531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:13.551975 9531 raft_consensus.cc:686] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [term 1 LEADER]: Becoming Leader. State: Replica: b30027368aba47a2954a87779a49d2a5, State: Running, Role: LEADER
I20260925 00:43:13.552385 9531 consensus_queue.cc:237] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30027368aba47a2954a87779a49d2a5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36295 } }
I20260925 00:43:13.552839 9526 sys_catalog.cc:564] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:13.553165 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:13.539093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36656 (local address 127.6.72.126:36295)
0925 00:43:13.539533 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:13.539549 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:13.539585 (+ 36us) server_negotiation.cc:407] Connection header received
0925 00:43:13.540341 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:13.540360 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:13.540703 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:13.541006 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:13.541952 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:13.543014 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:13.544178 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:13.544715 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:13.544903 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:13.547419 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:13.547424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:13.547436 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:13.547462 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:13.549993 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:13.550464 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:13.550470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:13.550476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:13.550598 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:13.550897 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:13.550901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:13.550903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:13.551367 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:13.551563 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:13.551881 (+ 318us) server_negotiation.cc:299] Negotiation successful
0925 00:43:13.552179 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":153,"threads_started":1}
I20260925 00:43:13.553642 9531 sys_catalog.cc:455] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b30027368aba47a2954a87779a49d2a5. Latest consensus state: current_term: 1 leader_uuid: "b30027368aba47a2954a87779a49d2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30027368aba47a2954a87779a49d2a5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36295 } } }
I20260925 00:43:13.553854 9531 sys_catalog.cc:458] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:13.554059 9532 sys_catalog.cc:455] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b30027368aba47a2954a87779a49d2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b30027368aba47a2954a87779a49d2a5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36295 } } }
I20260925 00:43:13.554152 9532 sys_catalog.cc:458] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:13.554374 9540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:13.557605 9540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:13.563323 9540 catalog_manager.cc:1347] Generated new cluster ID: e008fb6ff25347e9be764e7b891ff493
I20260925 00:43:13.563423 9540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:13.581620 9540 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:13.582952 9540 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:13.590596 9540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b30027368aba47a2954a87779a49d2a5: Generated new TSK 0
I20260925 00:43:13.591457 9540 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:43:13.624722 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36295
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:13.738535 9552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:13.738847 9552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:13.738927 9552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:13.742988 9552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:13.743094 9552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:13.743194 9552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:13.748519 9552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:13.749681 9552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:13.750775 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:13.760795 9557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:13.760829 9558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:13.760823 9560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:13.760915 9552 server_base.cc:1034] running on GCE node
I20260925 00:43:13.761576 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:13.762241 9552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:13.763437 9552 hybrid_clock.cc:648] HybridClock initialized: now 1790296993763417 us; error 32 us; skew 500 ppm
Sep 25 00:43:13 dist-test-slave-w8nh krb5kdc[9445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:13.766160 9552 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:43:13.767046 9552 webserver.cc:460] Webserver started at http://127.6.72.65:38143/ using document root <none> and password file <none>
I20260925 00:43:13.767621 9552 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:13.767712 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:13.767946 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:13.770519 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "493a036d481d4824a7eca61a3ffd5481"
format_stamp: "Formatted at 2026-09-25 00:43:13 on dist-test-slave-w8nh"
server_key: "abc9e79a298f793242b1f859badcef22"
server_key_iv: "ceebbcb31582c62b73ecae3ffc3a48a8"
server_key_version: "encryptionkey@0"
I20260925 00:43:13.771152 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "493a036d481d4824a7eca61a3ffd5481"
format_stamp: "Formatted at 2026-09-25 00:43:13 on dist-test-slave-w8nh"
server_key: "abc9e79a298f793242b1f859badcef22"
server_key_iv: "ceebbcb31582c62b73ecae3ffc3a48a8"
server_key_version: "encryptionkey@0"
I20260925 00:43:13.775003 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 00:43:13.777400 9567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:13.778594 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:13.778726 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "493a036d481d4824a7eca61a3ffd5481"
format_stamp: "Formatted at 2026-09-25 00:43:13 on dist-test-slave-w8nh"
server_key: "abc9e79a298f793242b1f859badcef22"
server_key_iv: "ceebbcb31582c62b73ecae3ffc3a48a8"
server_key_version: "encryptionkey@0"
I20260925 00:43:13.778839 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:13.803136 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:13.806824 9552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:13.807040 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:13.807662 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:13.808583 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:13.808653 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:13.808735 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:13.808785 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:13.817785 9552 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:45903
I20260925 00:43:13.817809 9680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:45903 every 8 connection(s)
I20260925 00:43:13.818708 9552 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:13.821996 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 9552
I20260925 00:43:13.822132 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:13.822496 6433 external_mini_cluster.cc:1428] Setting key 81e3cdb003a55318689bd27390f6c508
Sep 25 00:43:13 dist-test-slave-w8nh krb5kdc[9445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:43:13.834187 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:13.820578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:39031 (local address 127.6.72.126:36295)
0925 00:43:13.820716 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:13.820720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:13.821587 (+ 867us) server_negotiation.cc:407] Connection header received
0925 00:43:13.823378 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:13.823381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:13.823433 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:13.823538 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:13.825150 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:13.825840 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:13.827034 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:13.827300 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:13.827386 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:13.830466 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:13.830469 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:13.830471 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:13.830496 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:13.831975 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:13.832638 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:13.832641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:13.832643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:13.832724 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:13.833068 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:13.833071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:13.833073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:13.833262 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:13.833370 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:13.833880 (+ 510us) server_negotiation.cc:299] Negotiation successful
0925 00:43:13.834010 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 00:43:13.835012 9683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:13.820996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36295 (local address 127.6.72.65:39031)
0925 00:43:13.821437 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:13.821481 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:13.823166 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:13.823719 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:13.823726 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:13.824234 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:13.824989 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:13.825010 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:13.825993 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:13.825996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:13.826910 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:13.826919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:13.827478 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:13.827485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:13.827723 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:13.828512 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:13.828539 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:13.830285 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:13.832113 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:13.832119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:13.832136 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:13.832523 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:13.832815 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:13.832820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:13.832822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:13.832966 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:13.833369 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:13.833376 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:13.833639 (+ 263us) client_negotiation.cc:769] Sending connection context
0925 00:43:13.833826 (+ 187us) client_negotiation.cc:240] Negotiation successful
0925 00:43:13.834157 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:43:13.836308 9681 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36295
I20260925 00:43:13.836581 9681 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:13.837100 9681 heartbeater.cc:511] Master 127.6.72.126:36295 requested a full tablet report, sending...
I20260925 00:43:13.838593 9490 ts_manager.cc:194] Registered new tserver with Master: 493a036d481d4824a7eca61a3ffd5481 (127.6.72.65:45903)
I20260925 00:43:13.839723 9490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:39031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:43:13.852231 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36295
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:13.967772 9686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:13.968144 9686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:13.968220 9686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:13.972675 9686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:13.972762 9686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:13.972826 9686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:13.978629 9686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:13.979803 9686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:13.981027 9686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:13.991459 9691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:13.991466 9692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:13.991458 9694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:13.991830 9686 server_base.cc:1034] running on GCE node
I20260925 00:43:13.992424 9686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:13.993113 9686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:13.994320 9686 hybrid_clock.cc:648] HybridClock initialized: now 1790296993994301 us; error 26 us; skew 500 ppm
Sep 25 00:43:13 dist-test-slave-w8nh krb5kdc[9445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:13.997929 9686 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:43:13.998893 9686 webserver.cc:460] Webserver started at http://127.6.72.66:44009/ using document root <none> and password file <none>
I20260925 00:43:13.999506 9686 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:13.999570 9686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:13.999742 9686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:14.002118 9686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "eff05030abd341a991f1fa23b65b1171"
format_stamp: "Formatted at 2026-09-25 00:43:13 on dist-test-slave-w8nh"
server_key: "8d4fdcfd42f0f11f751af344a8c3c3ee"
server_key_iv: "d696bd30ee0d9dfe054f05cf58b2b00f"
server_key_version: "encryptionkey@0"
I20260925 00:43:14.002655 9686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "eff05030abd341a991f1fa23b65b1171"
format_stamp: "Formatted at 2026-09-25 00:43:13 on dist-test-slave-w8nh"
server_key: "8d4fdcfd42f0f11f751af344a8c3c3ee"
server_key_iv: "d696bd30ee0d9dfe054f05cf58b2b00f"
server_key_version: "encryptionkey@0"
I20260925 00:43:14.006562 9686 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260925 00:43:14.009124 9701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:14.010409 9686 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:43:14.010550 9686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "eff05030abd341a991f1fa23b65b1171"
format_stamp: "Formatted at 2026-09-25 00:43:13 on dist-test-slave-w8nh"
server_key: "8d4fdcfd42f0f11f751af344a8c3c3ee"
server_key_iv: "d696bd30ee0d9dfe054f05cf58b2b00f"
server_key_version: "encryptionkey@0"
I20260925 00:43:14.010661 9686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:14.019935 9686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:14.023487 9686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:14.023702 9686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:14.024417 9686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:14.025326 9686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:14.025372 9686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:14.025411 9686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:14.025426 9686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:14.034552 9686 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:43527
I20260925 00:43:14.034574 9814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:43527 every 8 connection(s)
I20260925 00:43:14.035537 9686 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:14.039449 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 9686
I20260925 00:43:14.039562 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:14.039858 6433 external_mini_cluster.cc:1428] Setting key a765f6d768dadb355f30d96e82e9e9c4
Sep 25 00:43:14 dist-test-slave-w8nh krb5kdc[9445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296993, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:43:14.051029 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:14.037548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:45919 (local address 127.6.72.126:36295)
0925 00:43:14.037739 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:14.037745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:14.038612 (+ 867us) server_negotiation.cc:407] Connection header received
0925 00:43:14.040134 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:14.040138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:14.040207 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:14.040348 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:14.041816 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:14.042360 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:14.043536 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:14.043767 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:14.043875 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:14.046925 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:14.046929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:14.046931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:14.046958 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:14.048753 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:14.049401 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:14.049404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:14.049405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:14.049487 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:14.049858 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:14.049861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:14.049863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:14.050064 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:14.050175 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:14.050722 (+ 547us) server_negotiation.cc:299] Negotiation successful
0925 00:43:14.050857 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:43:14.051977 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:14.037964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36295 (local address 127.6.72.66:45919)
0925 00:43:14.038446 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:14.038492 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:14.039877 (+ 1385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:14.040584 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:14.040594 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:14.041055 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:14.041669 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:14.041684 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:14.042490 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:14.042493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:14.043403 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:14.043413 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:14.043933 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:14.043936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:14.044107 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:14.044811 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:14.044830 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:14.046718 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:14.048903 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:14.048909 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:14.048927 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:14.049219 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:14.049585 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:14.049589 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:14.049592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:14.049748 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:14.050179 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:14.050189 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:14.050458 (+ 269us) client_negotiation.cc:769] Sending connection context
0925 00:43:14.050672 (+ 214us) client_negotiation.cc:240] Negotiation successful
0925 00:43:14.051039 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":125,"threads_started":1}
I20260925 00:43:14.053581 9815 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36295
I20260925 00:43:14.053880 9815 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:14.054512 9815 heartbeater.cc:511] Master 127.6.72.126:36295 requested a full tablet report, sending...
I20260925 00:43:14.055732 9490 ts_manager.cc:194] Registered new tserver with Master: eff05030abd341a991f1fa23b65b1171 (127.6.72.66:43527)
I20260925 00:43:14.056665 9490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:45919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:43:14.069502 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36295
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:14.181944 9820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:14.182225 9820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:14.182276 9820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:14.186479 9820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:14.186542 9820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:14.186601 9820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:14.191879 9820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:14.192951 9820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:14.193917 9820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:14.203452 9825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:14.203449 9828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:14.203516 9826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:14.203785 9820 server_base.cc:1034] running on GCE node
I20260925 00:43:14.204131 9820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:14.204823 9820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:14.205986 9820 hybrid_clock.cc:648] HybridClock initialized: now 1790296994205950 us; error 46 us; skew 500 ppm
Sep 25 00:43:14 dist-test-slave-w8nh krb5kdc[9445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296994, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:14.208738 9820 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:43:14.209540 9820 webserver.cc:460] Webserver started at http://127.6.72.67:39187/ using document root <none> and password file <none>
I20260925 00:43:14.210049 9820 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:14.210108 9820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:14.210282 9820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:14.212222 9820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "06991f80295e43e69ac6332bfb68a9db"
format_stamp: "Formatted at 2026-09-25 00:43:14 on dist-test-slave-w8nh"
server_key: "91723d67ca86fea9ac24c2658bdec1ec"
server_key_iv: "d3727eb72f2c3580c3170030a1929541"
server_key_version: "encryptionkey@0"
I20260925 00:43:14.212826 9820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "06991f80295e43e69ac6332bfb68a9db"
format_stamp: "Formatted at 2026-09-25 00:43:14 on dist-test-slave-w8nh"
server_key: "91723d67ca86fea9ac24c2658bdec1ec"
server_key_iv: "d3727eb72f2c3580c3170030a1929541"
server_key_version: "encryptionkey@0"
I20260925 00:43:14.216640 9820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:43:14.219079 9835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:14.220368 9820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:14.220502 9820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "06991f80295e43e69ac6332bfb68a9db"
format_stamp: "Formatted at 2026-09-25 00:43:14 on dist-test-slave-w8nh"
server_key: "91723d67ca86fea9ac24c2658bdec1ec"
server_key_iv: "d3727eb72f2c3580c3170030a1929541"
server_key_version: "encryptionkey@0"
I20260925 00:43:14.220620 9820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:14.243799 9820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:14.247292 9820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:14.247503 9820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:14.248106 9820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:14.249013 9820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:14.249083 9820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:14.249152 9820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:14.249204 9820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:14.258285 9820 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:38657
I20260925 00:43:14.258334 9948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:38657 every 8 connection(s)
I20260925 00:43:14.259208 9820 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:14.266357 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 9820
I20260925 00:43:14.266502 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:14.266851 6433 external_mini_cluster.cc:1428] Setting key bb58174de0acd483860ee84fa1f4ebc6
Sep 25 00:43:14 dist-test-slave-w8nh krb5kdc[9445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296994, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:14.273908 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:14.261079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:59595 (local address 127.6.72.126:36295)
0925 00:43:14.261228 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:14.261233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:14.262154 (+ 921us) server_negotiation.cc:407] Connection header received
0925 00:43:14.263542 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:14.263544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:14.263590 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:14.263669 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:14.264976 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:14.265649 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:14.267275 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:14.267560 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:14.267680 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:14.270515 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:14.270518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:14.270521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:14.270555 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:14.271789 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:14.272375 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:14.272379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:14.272381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:14.272466 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:14.272815 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:14.272818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:14.272820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:14.273003 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:14.273108 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:14.273607 (+ 499us) server_negotiation.cc:299] Negotiation successful
0925 00:43:14.273726 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:43:14.274698 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:14.261529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36295 (local address 127.6.72.67:59595)
0925 00:43:14.261987 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:14.262032 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:14.263346 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:14.263850 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:14.263857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:14.264315 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:14.264837 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:14.264852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:14.265774 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:14.265777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:14.267114 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:14.267126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:14.267711 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:14.267737 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:14.267978 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:14.268718 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:14.268735 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:14.270346 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:14.271914 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:14.271919 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:14.271935 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:14.272192 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:14.272560 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:14.272563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:14.272565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:14.272707 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:14.273104 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:14.273110 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:14.273367 (+ 257us) client_negotiation.cc:769] Sending connection context
0925 00:43:14.273551 (+ 184us) client_negotiation.cc:240] Negotiation successful
0925 00:43:14.273880 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":108,"threads_started":1}
I20260925 00:43:14.275801 9949 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36295
I20260925 00:43:14.276007 9949 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:14.276567 9949 heartbeater.cc:511] Master 127.6.72.126:36295 requested a full tablet report, sending...
I20260925 00:43:14.277539 9490 ts_manager.cc:194] Registered new tserver with Master: 06991f80295e43e69ac6332bfb68a9db (127.6.72.67:38657)
I20260925 00:43:14.278048 9490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:59595
I20260925 00:43:14.280786 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 00:43:14 dist-test-slave-w8nh krb5kdc[9445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296994, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 00:43:14 dist-test-slave-w8nh krb5kdc[9445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296994, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:14.306492 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:14.297895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.6.72.126:36295)
0925 00:43:14.298071 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:14.298076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:14.298212 (+ 136us) server_negotiation.cc:407] Connection header received
0925 00:43:14.298382 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:14.298385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:14.298434 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:14.298521 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:14.299235 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:14.299763 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:14.300669 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:14.300904 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:14.301017 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:14.303382 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:14.303385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:14.303387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:14.303413 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:14.304811 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:14.305225 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:14.305228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:14.305230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:14.305315 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:14.305584 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:14.305588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:14.305590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:14.305825 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:14.305970 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:14.306207 (+ 237us) server_negotiation.cc:299] Negotiation successful
0925 00:43:14.306322 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260925 00:43:14.306972 9490 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36664
I20260925 00:43:14.307962 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 9552
I20260925 00:43:14.313894 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 9686
I20260925 00:43:14.319145 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 9820
I20260925 00:43:14.324441 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 9458
2026-09-25T00:43:14Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2161 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:14 dist-test-slave-w8nh krb5kdc[9961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:14 dist-test-slave-w8nh krb5kdc[9961](info): set up 2 sockets
Sep 25 00:43:14 dist-test-slave-w8nh krb5kdc[9961](info): commencing operation
krb5kdc: starting...
W20260925 00:43:15.364724 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:15 dist-test-slave-w8nh krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:43:15.492162 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:36389
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:35241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:36389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:15.603596 9974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:15.603914 9974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:15.603958 9974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:15.608134 9974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:15.608212 9974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:15.608232 9974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:15.608268 9974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:15.608286 9974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:15.614017 9974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:36389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:36389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:15.615211 9974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:15.616291 9974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:15.625408 9982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:15.625656 9974 server_base.cc:1034] running on GCE node
W20260925 00:43:15.625408 9980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:15.625423 9979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:15.626477 9974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:15.627764 9974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:15.628957 9974 hybrid_clock.cc:648] HybridClock initialized: now 1790296995628944 us; error 29 us; skew 500 ppm
Sep 25 00:43:15 dist-test-slave-w8nh krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296995, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:15.631970 9974 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:43:15.633001 9974 webserver.cc:460] Webserver started at http://127.6.72.126:41619/ using document root <none> and password file <none>
I20260925 00:43:15.633554 9974 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:15.633628 9974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:15.633807 9974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:15.635918 9974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd"
format_stamp: "Formatted at 2026-09-25 00:43:15 on dist-test-slave-w8nh"
server_key: "3f2005479c73c0f4186b937ccb6dfc79"
server_key_iv: "370873c12f8117a6101f537924270cc4"
server_key_version: "encryptionkey@0"
I20260925 00:43:15.636554 9974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd"
format_stamp: "Formatted at 2026-09-25 00:43:15 on dist-test-slave-w8nh"
server_key: "3f2005479c73c0f4186b937ccb6dfc79"
server_key_iv: "370873c12f8117a6101f537924270cc4"
server_key_version: "encryptionkey@0"
I20260925 00:43:15.640926 9974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 00:43:15.643668 9989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:15.645179 9974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:15.645339 9974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd"
format_stamp: "Formatted at 2026-09-25 00:43:15 on dist-test-slave-w8nh"
server_key: "3f2005479c73c0f4186b937ccb6dfc79"
server_key_iv: "370873c12f8117a6101f537924270cc4"
server_key_version: "encryptionkey@0"
I20260925 00:43:15.645474 9974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:15.659539 9974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:15.662781 9974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:15.663000 9974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:15.670397 9974 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:36389
I20260925 00:43:15.670424 10041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:36389 every 8 connection(s)
I20260925 00:43:15.671420 9974 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:15.674314 10042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:15.679549 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 9974
I20260925 00:43:15.679674 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:15.679975 6433 external_mini_cluster.cc:1428] Setting key 150a2f6db659eade3241b956e147d653
I20260925 00:43:15.680356 10042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd: Bootstrap starting.
I20260925 00:43:15.683110 10042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:15.684031 10042 log.cc:824] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:15.686008 10042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd: No bootstrap required, opened a new log
Sep 25 00:43:15 dist-test-slave-w8nh krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:15.689921 10042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36389 } }
I20260925 00:43:15.690220 10042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:15.690286 10042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a25af97f97d42cda9c9a2bb51d5d1bd, State: Initialized, Role: FOLLOWER
I20260925 00:43:15.690852 10042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36389 } }
I20260925 00:43:15.691040 10042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:15.691136 10042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:15.691237 10042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:15.692160 10042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36389 } }
I20260925 00:43:15.692535 10042 leader_election.cc:302] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a25af97f97d42cda9c9a2bb51d5d1bd; no voters:
I20260925 00:43:15.692924 10042 leader_election.cc:288] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:15.692982 10047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:15.693166 10047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [term 1 LEADER]: Becoming Leader. State: Replica: 5a25af97f97d42cda9c9a2bb51d5d1bd, State: Running, Role: LEADER
I20260925 00:43:15.693531 10047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36389 } }
I20260925 00:43:15.694314 10042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:15.695308 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:15.681519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34640 (local address 127.6.72.126:36389)
0925 00:43:15.681978 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:15.681997 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:15.682038 (+ 41us) server_negotiation.cc:407] Connection header received
0925 00:43:15.682885 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:15.682909 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:15.683379 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:15.683753 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:15.684593 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:15.685580 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:15.686524 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:15.687006 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:15.687207 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:15.689830 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:15.689835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:15.689850 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:15.689877 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:15.691702 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:15.692182 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:15.692188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:15.692192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:15.692381 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:15.692679 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:15.692684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:15.692686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:15.693222 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:15.693967 (+ 745us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:15.694086 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 00:43:15.694332 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":128,"threads_started":1}
I20260925 00:43:15.695487 10049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36389 } } }
I20260925 00:43:15.695617 10049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:15.695923 10048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a25af97f97d42cda9c9a2bb51d5d1bd. Latest consensus state: current_term: 1 leader_uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a25af97f97d42cda9c9a2bb51d5d1bd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36389 } } }
I20260925 00:43:15.696019 10048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:15.696475 10056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:15.699501 10056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:15.705377 10056 catalog_manager.cc:1347] Generated new cluster ID: 20a7b60fb3c644979e9284a1df0496b2
I20260925 00:43:15.705466 10056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:15.738792 10056 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:15.740136 10056 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:15.758586 10056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a25af97f97d42cda9c9a2bb51d5d1bd: Generated new TSK 0
I20260925 00:43:15.759451 10056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:43:15.795845 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36389
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:35241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:15.919646 10068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:15.919950 10068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:15.919996 10068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:15.924357 10068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:15.924427 10068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:15.924489 10068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:15.929898 10068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:15.931056 10068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:15.932130 10068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:15.942147 10076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:15.942098 10074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:15.942209 10073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:15.942327 10068 server_base.cc:1034] running on GCE node
I20260925 00:43:15.942864 10068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:15.943511 10068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:15.944720 10068 hybrid_clock.cc:648] HybridClock initialized: now 1790296995944683 us; error 50 us; skew 500 ppm
Sep 25 00:43:15 dist-test-slave-w8nh krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296995, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:15.947752 10068 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:43:15.948786 10068 webserver.cc:460] Webserver started at http://127.6.72.65:45373/ using document root <none> and password file <none>
I20260925 00:43:15.949441 10068 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:15.949544 10068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:15.949800 10068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:15.952041 10068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "c702f489c41a4a0ab7e1a7635352592d"
format_stamp: "Formatted at 2026-09-25 00:43:15 on dist-test-slave-w8nh"
server_key: "68b87970c9048ad6220f3bf8d1e62663"
server_key_iv: "17596d2a668bb851264fb1bb5c6f6dc8"
server_key_version: "encryptionkey@0"
I20260925 00:43:15.952705 10068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "c702f489c41a4a0ab7e1a7635352592d"
format_stamp: "Formatted at 2026-09-25 00:43:15 on dist-test-slave-w8nh"
server_key: "68b87970c9048ad6220f3bf8d1e62663"
server_key_iv: "17596d2a668bb851264fb1bb5c6f6dc8"
server_key_version: "encryptionkey@0"
I20260925 00:43:15.956938 10068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:43:15.959898 10083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:15.961382 10068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:15.961506 10068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "c702f489c41a4a0ab7e1a7635352592d"
format_stamp: "Formatted at 2026-09-25 00:43:15 on dist-test-slave-w8nh"
server_key: "68b87970c9048ad6220f3bf8d1e62663"
server_key_iv: "17596d2a668bb851264fb1bb5c6f6dc8"
server_key_version: "encryptionkey@0"
I20260925 00:43:15.961609 10068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:15.983157 10068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:15.986867 10068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:15.987090 10068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:15.987766 10068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:15.988667 10068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:15.988713 10068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:15.988791 10068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:15.988822 10068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:15.998387 10068 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:38327
I20260925 00:43:15.998410 10196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:38327 every 8 connection(s)
I20260925 00:43:15.999356 10068 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:16.003969 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 10068
I20260925 00:43:16.004119 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:16.004483 6433 external_mini_cluster.cc:1428] Setting key 4292535ae32ea0fc082511d2fbcc0c49
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296995, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:43:16.016068 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:16.001754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:60671 (local address 127.6.72.126:36389)
0925 00:43:16.001913 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:16.001918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:16.003094 (+ 1176us) server_negotiation.cc:407] Connection header received
0925 00:43:16.005066 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:16.005072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:16.005132 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:16.005230 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:16.006609 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.007183 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.008494 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.008817 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.008962 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:16.011907 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:16.011911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:16.011916 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:16.011956 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:16.013814 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:16.014393 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:16.014397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:16.014399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:16.014490 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:16.014848 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:16.014852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:16.014854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:16.015053 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:16.015165 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:16.015737 (+ 572us) server_negotiation.cc:299] Negotiation successful
0925 00:43:16.015883 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 00:43:16.017042 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:16.002254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36389 (local address 127.6.72.65:60671)
0925 00:43:16.002892 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:16.002955 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:16.004740 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:16.005399 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:16.005407 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:16.005879 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:16.006446 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:16.006460 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.007360 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.007363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:16.008359 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:16.008369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.009006 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.009011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:16.009226 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:16.009927 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:16.009945 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:16.011698 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:16.013959 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:16.013964 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:16.013979 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:16.014283 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:16.014597 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:16.014601 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:16.014603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:16.014749 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:16.015160 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:16.015167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:16.015476 (+ 309us) client_negotiation.cc:769] Sending connection context
0925 00:43:16.015694 (+ 218us) client_negotiation.cc:240] Negotiation successful
0925 00:43:16.016038 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":138,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:43:16.018356 10197 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36389
I20260925 00:43:16.018623 10197 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:16.019136 10197 heartbeater.cc:511] Master 127.6.72.126:36389 requested a full tablet report, sending...
I20260925 00:43:16.020859 10006 ts_manager.cc:194] Registered new tserver with Master: c702f489c41a4a0ab7e1a7635352592d (127.6.72.65:38327)
I20260925 00:43:16.022109 10006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:60671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:43:16.035652 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36389
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:35241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:16.150871 10202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:16.151201 10202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:16.151286 10202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:16.155596 10202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:16.155712 10202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:16.155817 10202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:16.161471 10202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:16.162643 10202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:16.163692 10202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:16.173547 10210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:16.173552 10207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:16.173539 10208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:16.173976 10202 server_base.cc:1034] running on GCE node
I20260925 00:43:16.174378 10202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:16.175048 10202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:16.176321 10202 hybrid_clock.cc:648] HybridClock initialized: now 1790296996176295 us; error 30 us; skew 500 ppm
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296996, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:16.179239 10202 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:43:16.180226 10202 webserver.cc:460] Webserver started at http://127.6.72.66:46613/ using document root <none> and password file <none>
I20260925 00:43:16.180881 10202 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:16.180976 10202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:16.181212 10202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:16.183295 10202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "b86902fb7fca49acbb37f0552c4a4a15"
format_stamp: "Formatted at 2026-09-25 00:43:16 on dist-test-slave-w8nh"
server_key: "d980cd08dc154c277ddfcbf3fc4d3464"
server_key_iv: "dcba5cbea805c4c2077b52b49e71b000"
server_key_version: "encryptionkey@0"
I20260925 00:43:16.183874 10202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "b86902fb7fca49acbb37f0552c4a4a15"
format_stamp: "Formatted at 2026-09-25 00:43:16 on dist-test-slave-w8nh"
server_key: "d980cd08dc154c277ddfcbf3fc4d3464"
server_key_iv: "dcba5cbea805c4c2077b52b49e71b000"
server_key_version: "encryptionkey@0"
I20260925 00:43:16.187726 10202 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260925 00:43:16.190407 10217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:16.191691 10202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:16.191857 10202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "b86902fb7fca49acbb37f0552c4a4a15"
format_stamp: "Formatted at 2026-09-25 00:43:16 on dist-test-slave-w8nh"
server_key: "d980cd08dc154c277ddfcbf3fc4d3464"
server_key_iv: "dcba5cbea805c4c2077b52b49e71b000"
server_key_version: "encryptionkey@0"
I20260925 00:43:16.191979 10202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:16.223094 10202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:16.226619 10202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:16.226843 10202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:16.227468 10202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:16.228407 10202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:16.228477 10202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:16.228554 10202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:16.228600 10202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:16.238062 10202 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:45531
I20260925 00:43:16.238080 10330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:45531 every 8 connection(s)
I20260925 00:43:16.239018 10202 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:16.243059 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 10202
I20260925 00:43:16.243163 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:16.243438 6433 external_mini_cluster.cc:1428] Setting key f3aae722f63f660d57f5e1d9d6671e4e
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296996, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:43:16.254479 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:16.240926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:47065 (local address 127.6.72.126:36389)
0925 00:43:16.241089 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:16.241137 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:16.241850 (+ 713us) server_negotiation.cc:407] Connection header received
0925 00:43:16.243326 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:16.243330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:16.243392 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:16.243472 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:16.245047 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.245584 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.247024 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.247294 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.247423 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:16.250261 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:16.250266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:16.250270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:16.250311 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:16.251980 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:16.252749 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:16.252753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:16.252755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:16.252848 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:16.253251 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:16.253255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:16.253257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:16.253461 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:16.253575 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:16.254149 (+ 574us) server_negotiation.cc:299] Negotiation successful
0925 00:43:16.254292 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 00:43:16.255350 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:16.241232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36389 (local address 127.6.72.66:47065)
0925 00:43:16.241686 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:16.241733 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:16.243020 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:16.243645 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:16.243652 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:16.244136 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:16.244866 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:16.244888 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.245731 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.245735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:16.246885 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:16.246895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.247470 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.247474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:16.247665 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:16.248404 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:16.248422 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:16.250056 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:16.252279 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:16.252286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:16.252307 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:16.252626 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:16.252952 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:16.252956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:16.252958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:16.253154 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:16.253576 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:16.253584 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:16.253876 (+ 292us) client_negotiation.cc:769] Sending connection context
0925 00:43:16.254078 (+ 202us) client_negotiation.cc:240] Negotiation successful
0925 00:43:16.254452 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:43:16.256669 10331 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36389
I20260925 00:43:16.256949 10331 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:16.257476 10331 heartbeater.cc:511] Master 127.6.72.126:36389 requested a full tablet report, sending...
I20260925 00:43:16.258653 10006 ts_manager.cc:194] Registered new tserver with Master: b86902fb7fca49acbb37f0552c4a4a15 (127.6.72.66:45531)
I20260925 00:43:16.259294 10006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:47065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:43:16.273686 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36389
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:35241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:16.388015 10336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:16.388362 10336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:16.388412 10336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:16.392678 10336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:16.392748 10336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:16.392808 10336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:16.398600 10336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:16.399766 10336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:16.400857 10336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:16.410765 10341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:16.410895 10336 server_base.cc:1034] running on GCE node
W20260925 00:43:16.410792 10344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:16.410799 10342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:16.411484 10336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:16.412151 10336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:16.413314 10336 hybrid_clock.cc:648] HybridClock initialized: now 1790296996413295 us; error 29 us; skew 500 ppm
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296996, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:16.416093 10336 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:43:16.416998 10336 webserver.cc:460] Webserver started at http://127.6.72.67:44935/ using document root <none> and password file <none>
I20260925 00:43:16.417572 10336 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:16.417634 10336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:16.417877 10336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:16.419914 10336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "8cd2cd6a8b8241299c0357972884723f"
format_stamp: "Formatted at 2026-09-25 00:43:16 on dist-test-slave-w8nh"
server_key: "0eb1dc6c95c5ad22dfa6620414d098ef"
server_key_iv: "7084a8ce9eefe74397c2f13795a6d927"
server_key_version: "encryptionkey@0"
I20260925 00:43:16.420517 10336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "8cd2cd6a8b8241299c0357972884723f"
format_stamp: "Formatted at 2026-09-25 00:43:16 on dist-test-slave-w8nh"
server_key: "0eb1dc6c95c5ad22dfa6620414d098ef"
server_key_iv: "7084a8ce9eefe74397c2f13795a6d927"
server_key_version: "encryptionkey@0"
I20260925 00:43:16.424762 10336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 00:43:16.427240 10351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:16.428630 10336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:16.428766 10336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "8cd2cd6a8b8241299c0357972884723f"
format_stamp: "Formatted at 2026-09-25 00:43:16 on dist-test-slave-w8nh"
server_key: "0eb1dc6c95c5ad22dfa6620414d098ef"
server_key_iv: "7084a8ce9eefe74397c2f13795a6d927"
server_key_version: "encryptionkey@0"
I20260925 00:43:16.428884 10336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:16.443809 10336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:16.447034 10336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:16.447268 10336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:16.447888 10336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:16.448830 10336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:16.448920 10336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:16.448993 10336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:16.449044 10336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:16.458230 10336 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:41817
I20260925 00:43:16.458269 10464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:41817 every 8 connection(s)
I20260925 00:43:16.459209 10336 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:16.460602 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 10336
I20260925 00:43:16.460752 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:16.461091 6433 external_mini_cluster.cc:1428] Setting key 249bf646bfef8708f58c482e3efab2c5
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296996, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:16.473811 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:16.461274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:56893 (local address 127.6.72.126:36389)
0925 00:43:16.461431 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:16.461437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:16.462382 (+ 945us) server_negotiation.cc:407] Connection header received
0925 00:43:16.464081 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:16.464084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:16.464139 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:16.464216 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:16.465530 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.466156 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.467296 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.467516 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.467635 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:16.470240 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:16.470245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:16.470252 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:16.470279 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:16.471637 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:16.472177 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:16.472182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:16.472184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:16.472318 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:16.472679 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:16.472682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:16.472684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:16.472875 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:16.472984 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:16.473502 (+ 518us) server_negotiation.cc:299] Negotiation successful
0925 00:43:16.473639 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 00:43:16.474921 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:16.461679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36389 (local address 127.6.72.67:56893)
0925 00:43:16.462229 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:16.462280 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:16.463871 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:16.464413 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:16.464420 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:16.464868 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:16.465388 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:16.465403 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.466286 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.466289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:16.467163 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:16.467172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.467669 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.467673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:16.467851 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:16.468507 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:16.468524 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:16.469975 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:16.471767 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:16.471773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:16.471791 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:16.472066 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:16.472412 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:16.472415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:16.472417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:16.472568 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:16.472982 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:16.472989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:16.473262 (+ 273us) client_negotiation.cc:769] Sending connection context
0925 00:43:16.473464 (+ 202us) client_negotiation.cc:240] Negotiation successful
0925 00:43:16.473812 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":131,"threads_started":1}
I20260925 00:43:16.476202 10465 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36389
I20260925 00:43:16.476523 10465 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:16.477057 10465 heartbeater.cc:511] Master 127.6.72.126:36389 requested a full tablet report, sending...
I20260925 00:43:16.478161 10006 ts_manager.cc:194] Registered new tserver with Master: 8cd2cd6a8b8241299c0357972884723f (127.6.72.67:41817)
I20260925 00:43:16.478757 10006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:56893
I20260925 00:43:16.487737 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[9961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:43:16.513913 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:16.504803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55270 (local address 127.6.72.65:38327)
0925 00:43:16.505163 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:16.505180 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:16.505205 (+ 25us) server_negotiation.cc:407] Connection header received
0925 00:43:16.505262 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:16.505267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:16.505472 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:16.505608 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:16.506256 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.507019 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.507786 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.508182 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.508352 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:16.510673 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:16.510682 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:16.510694 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:16.510716 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:16.512222 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:16.512657 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:16.512662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:16.512663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:16.512739 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:16.512966 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:16.512968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:16.512969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:16.513228 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:16.513400 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:16.513550 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 00:43:16.513719 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":96,"threads_started":1}
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[9961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:16.525185 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:16.516824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.6.72.126:36389)
0925 00:43:16.516969 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:16.516972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:16.516987 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:43:16.517097 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:16.517099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:16.517139 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:16.517215 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:16.517841 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.518366 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.519117 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:16.519356 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:16.519453 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:16.522009 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:16.522012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:16.522014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:16.522040 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:16.523394 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:16.523788 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:16.523791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:16.523792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:16.523868 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:16.524092 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:16.524100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:16.524101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:16.524453 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:16.524561 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:16.524803 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 00:43:16.524953 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260925 00:43:16.525638 10006 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34654
I20260925 00:43:16.526304 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 10068
I20260925 00:43:16.532193 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 10202
I20260925 00:43:16.537513 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 10336
I20260925 00:43:16.542871 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 9974
2026-09-25T00:43:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2218 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[10477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[10477](info): set up 2 sockets
Sep 25 00:43:16 dist-test-slave-w8nh krb5kdc[10477](info): commencing operation
krb5kdc: starting...
W20260925 00:43:17.584684 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:17 dist-test-slave-w8nh krb5kdc[10477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:43:17.711994 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:40005
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:40005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:17.825110 10490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:17.825456 10490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:17.825534 10490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:17.829752 10490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:17.829852 10490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:17.829890 10490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:17.829957 10490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:17.829994 10490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:17.835604 10490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:40005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:40005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:17.836793 10490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:17.837854 10490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:17.846449 10495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:17.846443 10496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:17.846449 10498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:17.846732 10490 server_base.cc:1034] running on GCE node
I20260925 00:43:17.847410 10490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:17.848595 10490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:17.849763 10490 hybrid_clock.cc:648] HybridClock initialized: now 1790296997849733 us; error 44 us; skew 500 ppm
Sep 25 00:43:17 dist-test-slave-w8nh krb5kdc[10477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296997, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:17.852563 10490 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:43:17.853340 10490 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:17.853443 10490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:17.853682 10490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:17.855827 10490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "f697787f78974ef1ad6e7cd06e0d2cbd"
format_stamp: "Formatted at 2026-09-25 00:43:17 on dist-test-slave-w8nh"
server_key: "f8c42e270afe46796f9898893bbea37e"
server_key_iv: "9f1c2e49b1a606694f1cb48d3feae265"
server_key_version: "encryptionkey@0"
I20260925 00:43:17.856484 10490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "f697787f78974ef1ad6e7cd06e0d2cbd"
format_stamp: "Formatted at 2026-09-25 00:43:17 on dist-test-slave-w8nh"
server_key: "f8c42e270afe46796f9898893bbea37e"
server_key_iv: "9f1c2e49b1a606694f1cb48d3feae265"
server_key_version: "encryptionkey@0"
I20260925 00:43:17.860442 10490 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 00:43:17.862922 10504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:17.864171 10490 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:43:17.864363 10490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "f697787f78974ef1ad6e7cd06e0d2cbd"
format_stamp: "Formatted at 2026-09-25 00:43:17 on dist-test-slave-w8nh"
server_key: "f8c42e270afe46796f9898893bbea37e"
server_key_iv: "9f1c2e49b1a606694f1cb48d3feae265"
server_key_version: "encryptionkey@0"
I20260925 00:43:17.864491 10490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:17.874110 10490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:17.877391 10490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:17.877600 10490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:17.884429 10490 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:40005
I20260925 00:43:17.884696 10556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:40005 every 8 connection(s)
I20260925 00:43:17.885142 10490 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:17.887871 10557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:17.888219 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 10490
I20260925 00:43:17.888415 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:17.888677 6433 external_mini_cluster.cc:1428] Setting key d2ee040d20d46c5345b2b2a311948954
I20260925 00:43:17.894776 10557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd: Bootstrap starting.
I20260925 00:43:17.897578 10557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd: Neither blocks nor log segments found. Creating new log.
Sep 25 00:43:17 dist-test-slave-w8nh krb5kdc[10477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:17.898424 10557 log.cc:824] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:17.900862 10557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd: No bootstrap required, opened a new log
I20260925 00:43:17.903611 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:17.890119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59280 (local address 127.6.72.126:40005)
0925 00:43:17.890527 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:17.890544 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:17.890589 (+ 45us) server_negotiation.cc:407] Connection header received
0925 00:43:17.891278 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:17.891295 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:17.891631 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:17.891921 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:17.892854 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:17.893819 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:17.894959 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:17.895448 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:17.895568 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:17.898314 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:17.898321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:17.898339 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:17.898371 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:17.900288 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:17.900719 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:17.900725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:17.900730 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:17.900884 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:17.901178 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:17.901183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:17.901185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:17.901696 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:17.901899 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:17.902217 (+ 318us) server_negotiation.cc:299] Negotiation successful
0925 00:43:17.902532 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":126,"threads_started":1}
I20260925 00:43:17.905417 10557 raft_consensus.cc:348] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f697787f78974ef1ad6e7cd06e0d2cbd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40005 } }
I20260925 00:43:17.905663 10557 raft_consensus.cc:374] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:17.905735 10557 raft_consensus.cc:729] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f697787f78974ef1ad6e7cd06e0d2cbd, State: Initialized, Role: FOLLOWER
I20260925 00:43:17.906149 10557 consensus_queue.cc:260] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f697787f78974ef1ad6e7cd06e0d2cbd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40005 } }
I20260925 00:43:17.906302 10557 raft_consensus.cc:388] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:17.906395 10557 raft_consensus.cc:482] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:17.906512 10557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:17.907456 10557 raft_consensus.cc:504] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f697787f78974ef1ad6e7cd06e0d2cbd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40005 } }
I20260925 00:43:17.907804 10557 leader_election.cc:302] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f697787f78974ef1ad6e7cd06e0d2cbd; no voters:
I20260925 00:43:17.908118 10557 leader_election.cc:288] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:17.908381 10562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:17.908676 10562 raft_consensus.cc:686] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [term 1 LEADER]: Becoming Leader. State: Replica: f697787f78974ef1ad6e7cd06e0d2cbd, State: Running, Role: LEADER
I20260925 00:43:17.908993 10562 consensus_queue.cc:237] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f697787f78974ef1ad6e7cd06e0d2cbd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40005 } }
I20260925 00:43:17.909508 10557 sys_catalog.cc:564] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:17.910722 10563 sys_catalog.cc:455] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f697787f78974ef1ad6e7cd06e0d2cbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f697787f78974ef1ad6e7cd06e0d2cbd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40005 } } }
I20260925 00:43:17.910882 10563 sys_catalog.cc:458] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:17.911082 10564 sys_catalog.cc:455] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f697787f78974ef1ad6e7cd06e0d2cbd. Latest consensus state: current_term: 1 leader_uuid: "f697787f78974ef1ad6e7cd06e0d2cbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f697787f78974ef1ad6e7cd06e0d2cbd" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40005 } } }
I20260925 00:43:17.911237 10564 sys_catalog.cc:458] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:17.911993 10571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:17.914717 10571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:17.920503 10571 catalog_manager.cc:1347] Generated new cluster ID: 7689cf4ef3c747ee8809ef00f4150013
I20260925 00:43:17.920585 10571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:17.941560 10571 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:17.942572 10571 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:17.950429 10571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd: Generated new TSK 0
I20260925 00:43:17.951197 10571 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:43:17.985373 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40005
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:18.099879 10583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:18.100142 10583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:18.100189 10583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:18.104447 10583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:18.104512 10583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:18.104602 10583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:18.110106 10583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:18.111299 10583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:18.112375 10583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:18.121855 10591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:18.121876 10589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:18.121829 10588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:18.122056 10583 server_base.cc:1034] running on GCE node
I20260925 00:43:18.122586 10583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:18.123247 10583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:18.124432 10583 hybrid_clock.cc:648] HybridClock initialized: now 1790296998124421 us; error 31 us; skew 500 ppm
Sep 25 00:43:18 dist-test-slave-w8nh krb5kdc[10477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:18.127348 10583 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:43:18.128083 10583 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:18.128177 10583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:18.128453 10583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:18.130527 10583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b"
format_stamp: "Formatted at 2026-09-25 00:43:18 on dist-test-slave-w8nh"
server_key: "a083f82135e6d11ad04aae7d5bd7fbb4"
server_key_iv: "df908744f80f09fce7620df9e7fa3c2a"
server_key_version: "encryptionkey@0"
I20260925 00:43:18.131122 10583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b"
format_stamp: "Formatted at 2026-09-25 00:43:18 on dist-test-slave-w8nh"
server_key: "a083f82135e6d11ad04aae7d5bd7fbb4"
server_key_iv: "df908744f80f09fce7620df9e7fa3c2a"
server_key_version: "encryptionkey@0"
I20260925 00:43:18.134925 10583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 00:43:18.137492 10597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:18.138828 10583 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:18.138967 10583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b"
format_stamp: "Formatted at 2026-09-25 00:43:18 on dist-test-slave-w8nh"
server_key: "a083f82135e6d11ad04aae7d5bd7fbb4"
server_key_iv: "df908744f80f09fce7620df9e7fa3c2a"
server_key_version: "encryptionkey@0"
I20260925 00:43:18.139086 10583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:18.151957 10583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:18.155332 10583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:18.155540 10583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:18.156131 10583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:18.157017 10583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:18.157091 10583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:18.157164 10583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:18.157202 10583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:18.166491 10583 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:32861
I20260925 00:43:18.166508 10710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:32861 every 8 connection(s)
I20260925 00:43:18.167100 10583 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:18.172708 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 10583
I20260925 00:43:18.172837 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:18.173110 6433 external_mini_cluster.cc:1428] Setting key 8aa9d20b1fccfb30fa60845771fdd19e
Sep 25 00:43:18 dist-test-slave-w8nh krb5kdc[10477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:18.181701 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.168866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:48061 (local address 127.6.72.126:40005)
0925 00:43:18.169007 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:18.169011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:18.169919 (+ 908us) server_negotiation.cc:407] Connection header received
0925 00:43:18.171116 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:18.171119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:18.171167 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:18.171241 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:18.172443 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.173257 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.174601 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.174954 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.175077 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.177525 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:18.177532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:18.177538 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:18.177573 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:18.179446 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:18.179919 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:18.179925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:18.179929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:18.180034 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:18.180437 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:18.180443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:18.180446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:18.180731 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:18.180867 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:18.181345 (+ 478us) server_negotiation.cc:299] Negotiation successful
0925 00:43:18.181489 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 00:43:18.182430 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.169326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:40005 (local address 127.6.72.65:48061)
0925 00:43:18.169761 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:18.169804 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:18.170920 (+ 1116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:18.171403 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:18.171410 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:18.171762 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:18.172225 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.172298 (+ 73us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.173371 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.173373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.174447 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.174461 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.175094 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.175097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.175233 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.175893 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:18.175909 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:18.177368 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:18.179555 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:18.179559 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:18.179572 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:18.179816 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:18.180137 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:18.180140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:18.180141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:18.180342 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:18.180851 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:18.180856 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:18.181088 (+ 232us) client_negotiation.cc:769] Sending connection context
0925 00:43:18.181240 (+ 152us) client_negotiation.cc:240] Negotiation successful
0925 00:43:18.181501 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:43:18.183722 10711 heartbeater.cc:347] Connected to a master server at 127.6.72.126:40005
I20260925 00:43:18.183982 10711 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:18.184538 10711 heartbeater.cc:511] Master 127.6.72.126:40005 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:43:18.186093 10521 ts_manager.cc:194] Registered new tserver with Master: 0dba36eaaf7a4fb39a4c85cff6f4a85b (127.6.72.65:32861)
I20260925 00:43:18.187718 10521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:48061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:43:18.203552 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40005
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:18.316092 10716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:18.316447 10716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:18.316525 10716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:18.320652 10716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:18.320758 10716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:18.320878 10716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:18.326278 10716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:18.327433 10716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:18.328505 10716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:18.337984 10724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:18.338030 10722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:18.337984 10721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:18.338358 10716 server_base.cc:1034] running on GCE node
I20260925 00:43:18.338799 10716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:18.339484 10716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:18.340687 10716 hybrid_clock.cc:648] HybridClock initialized: now 1790296998340646 us; error 61 us; skew 500 ppm
Sep 25 00:43:18 dist-test-slave-w8nh krb5kdc[10477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:18.343530 10716 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:43:18.344383 10716 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:18.344476 10716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:18.344722 10716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:18.346787 10716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "8a243504a46e45eb91d9831335354979"
format_stamp: "Formatted at 2026-09-25 00:43:18 on dist-test-slave-w8nh"
server_key: "8b2c7bb369aaa9554e8a4d4af2585c38"
server_key_iv: "e4a9bc78afceda30854a00414516107a"
server_key_version: "encryptionkey@0"
I20260925 00:43:18.347354 10716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "8a243504a46e45eb91d9831335354979"
format_stamp: "Formatted at 2026-09-25 00:43:18 on dist-test-slave-w8nh"
server_key: "8b2c7bb369aaa9554e8a4d4af2585c38"
server_key_iv: "e4a9bc78afceda30854a00414516107a"
server_key_version: "encryptionkey@0"
I20260925 00:43:18.351109 10716 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 00:43:18.353487 10730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:18.354704 10716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:18.354840 10716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "8a243504a46e45eb91d9831335354979"
format_stamp: "Formatted at 2026-09-25 00:43:18 on dist-test-slave-w8nh"
server_key: "8b2c7bb369aaa9554e8a4d4af2585c38"
server_key_iv: "e4a9bc78afceda30854a00414516107a"
server_key_version: "encryptionkey@0"
I20260925 00:43:18.354955 10716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:18.369568 10716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:18.372573 10716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:18.372781 10716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:18.373387 10716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:18.374230 10716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:18.374302 10716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:18.374377 10716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:18.374433 10716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:18.383394 10716 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:34119
I20260925 00:43:18.383414 10843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:34119 every 8 connection(s)
I20260925 00:43:18.383993 10716 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:18.389767 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 10716
I20260925 00:43:18.389931 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:18.390219 6433 external_mini_cluster.cc:1428] Setting key a10651994380837f64a06760d8727612
Sep 25 00:43:18 dist-test-slave-w8nh krb5kdc[10477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:18.398691 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.385788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:60837 (local address 127.6.72.126:40005)
0925 00:43:18.385922 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:18.385925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:18.386732 (+ 807us) server_negotiation.cc:407] Connection header received
0925 00:43:18.388045 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:18.388049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:18.388110 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:18.388191 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:18.389413 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.390027 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.391376 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.391715 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.391836 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.394660 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:18.394664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:18.394666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:18.394693 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:18.396208 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:18.396942 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:18.396945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:18.396947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:18.397042 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:18.397488 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:18.397492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:18.397493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:18.397695 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:18.397783 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:18.398433 (+ 650us) server_negotiation.cc:299] Negotiation successful
0925 00:43:18.398548 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:43:18.399955 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.386154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:40005 (local address 127.6.72.66:60837)
0925 00:43:18.386584 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:18.386626 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:18.387829 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:18.388391 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:18.388398 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:18.388755 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:18.389246 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.389260 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.390182 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.390185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.391245 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.391255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.391875 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.391879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.392053 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.392728 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:18.392744 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:18.394473 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:18.396363 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:18.396373 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:18.396394 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:18.396811 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:18.397160 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:18.397165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:18.397170 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:18.397362 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:18.397811 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:18.397818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:18.398169 (+ 351us) client_negotiation.cc:769] Sending connection context
0925 00:43:18.398400 (+ 231us) client_negotiation.cc:240] Negotiation successful
0925 00:43:18.398786 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":117,"threads_started":1}
I20260925 00:43:18.401222 10844 heartbeater.cc:347] Connected to a master server at 127.6.72.126:40005
I20260925 00:43:18.401444 10844 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:18.401911 10844 heartbeater.cc:511] Master 127.6.72.126:40005 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:43:18.402951 10521 ts_manager.cc:194] Registered new tserver with Master: 8a243504a46e45eb91d9831335354979 (127.6.72.66:34119)
I20260925 00:43:18.403545 10521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:60837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:43:18.420454 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40005
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:18.531592 10849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:18.531867 10849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:18.531913 10849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:18.535949 10849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:18.536003 10849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:18.536063 10849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:18.541335 10849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:40005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:18.542318 10849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:18.543336 10849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:18.552538 10857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:18.552539 10855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:18.552538 10854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:18.552801 10849 server_base.cc:1034] running on GCE node
I20260925 00:43:18.553252 10849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:18.553839 10849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:18.555037 10849 hybrid_clock.cc:648] HybridClock initialized: now 1790296998555021 us; error 26 us; skew 500 ppm
Sep 25 00:43:18 dist-test-slave-w8nh krb5kdc[10477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:18.560756 10849 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:43:18.561525 10849 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:18.561619 10849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:18.561861 10849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:18.563930 10849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "9e8ae180203c4941bac4c73dc34db4ae"
format_stamp: "Formatted at 2026-09-25 00:43:18 on dist-test-slave-w8nh"
server_key: "42861c9458a976ffd093ab9315b079bb"
server_key_iv: "8645877ccdecd920785856399989abb4"
server_key_version: "encryptionkey@0"
I20260925 00:43:18.564518 10849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "9e8ae180203c4941bac4c73dc34db4ae"
format_stamp: "Formatted at 2026-09-25 00:43:18 on dist-test-slave-w8nh"
server_key: "42861c9458a976ffd093ab9315b079bb"
server_key_iv: "8645877ccdecd920785856399989abb4"
server_key_version: "encryptionkey@0"
I20260925 00:43:18.568428 10849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 00:43:18.570907 10863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:18.572543 10849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:18.572669 10849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "9e8ae180203c4941bac4c73dc34db4ae"
format_stamp: "Formatted at 2026-09-25 00:43:18 on dist-test-slave-w8nh"
server_key: "42861c9458a976ffd093ab9315b079bb"
server_key_iv: "8645877ccdecd920785856399989abb4"
server_key_version: "encryptionkey@0"
I20260925 00:43:18.572784 10849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:18.588729 10849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:18.591589 10849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:18.591794 10849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:18.592440 10849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:18.593288 10849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:18.593335 10849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:18.593407 10849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:18.593443 10849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:18.602381 10849 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:42025
I20260925 00:43:18.602406 10976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:42025 every 8 connection(s)
I20260925 00:43:18.602977 10849 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:18.606808 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 10849
I20260925 00:43:18.606907 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:18.607183 6433 external_mini_cluster.cc:1428] Setting key 68ac36be72835cd5fab981b93f9a5391
Sep 25 00:43:18 dist-test-slave-w8nh krb5kdc[10477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790296998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:18.617517 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.605093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:53035 (local address 127.6.72.126:40005)
0925 00:43:18.605255 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:18.605260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:18.606151 (+ 891us) server_negotiation.cc:407] Connection header received
0925 00:43:18.607492 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:18.607497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:18.607555 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:18.607628 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:18.609258 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.609861 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.611048 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.611263 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.611338 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.613912 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:18.613915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:18.613917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:18.613944 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:18.615587 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:18.616083 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:18.616086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:18.616087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:18.616162 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:18.616550 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:18.616552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:18.616553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:18.616732 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:18.616811 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:18.617278 (+ 467us) server_negotiation.cc:299] Negotiation successful
0925 00:43:18.617382 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:43:18.618338 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.605542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:40005 (local address 127.6.72.67:53035)
0925 00:43:18.605996 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:18.606039 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:18.607276 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:18.607838 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:18.607845 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:18.608391 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:18.609074 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.609092 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.609963 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.609966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.610941 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.610950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.611409 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.611414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.611567 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.612119 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:18.612136 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:18.613749 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:18.615711 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:18.615716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:18.615730 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:18.615994 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:18.616300 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:18.616306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:18.616309 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:18.616461 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:18.616821 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:18.616827 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:18.617064 (+ 237us) client_negotiation.cc:769] Sending connection context
0925 00:43:18.617212 (+ 148us) client_negotiation.cc:240] Negotiation successful
0925 00:43:18.617463 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":127,"threads_started":1}
I20260925 00:43:18.619515 10977 heartbeater.cc:347] Connected to a master server at 127.6.72.126:40005
I20260925 00:43:18.619794 10977 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:18.620398 10977 heartbeater.cc:511] Master 127.6.72.126:40005 requested a full tablet report, sending...
I20260925 00:43:18.621714 10521 ts_manager.cc:194] Registered new tserver with Master: 9e8ae180203c4941bac4c73dc34db4ae (127.6.72.67:42025)
I20260925 00:43:18.622241 10521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:53035
I20260925 00:43:18.633268 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:43:18.643244 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.635281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49534 (local address 127.6.72.126:40005)
0925 00:43:18.635442 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:18.635446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:18.635568 (+ 122us) server_negotiation.cc:407] Connection header received
0925 00:43:18.635735 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:18.635738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:18.635792 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:18.635876 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:18.636659 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.637281 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.638117 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.638356 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.638466 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.639876 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:18.639880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:18.639883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:18.639918 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:18.641466 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:18.641926 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:18.641930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:18.641932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:18.642018 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:18.642298 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:18.642302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:18.642303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:18.642528 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:18.642670 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:18.642939 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 00:43:18.643061 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 00:43:18.646418 10521 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:18.649668 10521 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:18.664706 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.658462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:32861 (local address 127.0.0.1:39498)
0925 00:43:18.659741 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:18.659797 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:18.659944 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:18.660750 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:18.660755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:18.660778 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:18.660880 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.660885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.662317 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.662323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.663120 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.663129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.664090 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.664094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.664209 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.664299 (+ 90us) client_negotiation.cc:769] Sending connection context
0925 00:43:18.664408 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 00:43:18.664528 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"spinlock_wait_cycles":768,"thread_start_us":98,"threads_started":1}
I20260925 00:43:18.664731 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.658561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39498 (local address 127.6.72.65:32861)
0925 00:43:18.659110 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:18.659119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:18.659883 (+ 764us) server_negotiation.cc:407] Connection header received
0925 00:43:18.660126 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:18.660136 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:18.660385 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:18.660518 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:18.661064 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.662148 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.663247 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.663958 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.664067 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.664214 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:18.664454 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 00:43:18.664560 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
I20260925 00:43:18.665114 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.659285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:42025 (local address 127.0.0.1:41600)
0925 00:43:18.659741 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:18.659797 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:18.659944 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:18.660698 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:18.660701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:18.660720 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:18.660845 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.660851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.661998 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.662001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.663466 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.663476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.664699 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.664702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.664800 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.664832 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 00:43:18.664907 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 00:43:18.664978 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":141,"spinlock_wait_cycles":6272,"thread_start_us":100,"threads_started":1}
I20260925 00:43:18.665640 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.659219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60784 (local address 127.6.72.66:34119)
0925 00:43:18.659905 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:18.659917 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:18.659939 (+ 22us) server_negotiation.cc:407] Connection header received
0925 00:43:18.660138 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:18.660145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:18.660373 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:18.660526 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:18.660849 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.662220 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.663661 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.664325 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.664429 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.664558 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:18.665397 (+ 839us) server_negotiation.cc:299] Negotiation successful
0925 00:43:18.665494 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":117,"threads_started":1}
I20260925 00:43:18.665650 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.659426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.6.72.67:42025)
0925 00:43:18.660135 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:18.660146 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:18.660165 (+ 19us) server_negotiation.cc:407] Connection header received
0925 00:43:18.660228 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:18.660232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:18.660423 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:18.660547 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:18.660970 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.661855 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.663595 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.664417 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.664515 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.664635 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:18.665386 (+ 751us) server_negotiation.cc:299] Negotiation successful
0925 00:43:18.665494 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":154,"threads_started":1}
I20260925 00:43:18.665302 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.658811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:34119 (local address 127.0.0.1:60784)
0925 00:43:18.659760 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:18.659797 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:18.659945 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:18.660533 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:18.660539 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:18.660562 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:18.660716 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.660725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.662351 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.662354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.663524 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.663534 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.664643 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.664646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.664741 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.664769 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 00:43:18.665079 (+ 310us) client_negotiation.cc:240] Negotiation successful
0925 00:43:18.665160 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":28,"spinlock_wait_cycles":9600,"thread_start_us":93,"threads_started":1}
I20260925 00:43:18.668128 10645 tablet_service.cc:1467] Processing CreateTablet for tablet 55f7a881dbfe4cf49b211d2fbe3b38c6 (DEFAULT_TABLE table=test-table [id=66c6a0d090eb4586897d81e89b006ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:18.668548 10911 tablet_service.cc:1467] Processing CreateTablet for tablet 55f7a881dbfe4cf49b211d2fbe3b38c6 (DEFAULT_TABLE table=test-table [id=66c6a0d090eb4586897d81e89b006ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:18.668548 10778 tablet_service.cc:1467] Processing CreateTablet for tablet 55f7a881dbfe4cf49b211d2fbe3b38c6 (DEFAULT_TABLE table=test-table [id=66c6a0d090eb4586897d81e89b006ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:18.669380 10645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f7a881dbfe4cf49b211d2fbe3b38c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:18.669553 10911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f7a881dbfe4cf49b211d2fbe3b38c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:18.669548 10778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f7a881dbfe4cf49b211d2fbe3b38c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:18.675782 10994 tablet_bootstrap.cc:492] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b: Bootstrap starting.
I20260925 00:43:18.678241 10995 tablet_bootstrap.cc:492] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: Bootstrap starting.
I20260925 00:43:18.678241 10996 tablet_bootstrap.cc:492] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: Bootstrap starting.
I20260925 00:43:18.678665 10994 tablet_bootstrap.cc:654] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:18.679742 10994 log.cc:824] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:18.680882 10995 tablet_bootstrap.cc:654] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:18.680882 10996 tablet_bootstrap.cc:654] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:18.681687 10994 tablet_bootstrap.cc:492] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b: No bootstrap required, opened a new log
I20260925 00:43:18.682051 10994 ts_tablet_manager.cc:1397] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:43:18.682207 10996 log.cc:824] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:18.682209 10995 log.cc:824] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:18.684185 10995 tablet_bootstrap.cc:492] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: No bootstrap required, opened a new log
I20260925 00:43:18.684419 10995 ts_tablet_manager.cc:1397] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:43:18.685614 10996 tablet_bootstrap.cc:492] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: No bootstrap required, opened a new log
I20260925 00:43:18.685822 10996 ts_tablet_manager.cc:1397] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260925 00:43:18.685973 10994 raft_consensus.cc:348] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 } } peers { permanent_uuid: "9e8ae180203c4941bac4c73dc34db4ae" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 42025 } } peers { permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 } }
I20260925 00:43:18.686254 10994 raft_consensus.cc:374] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:18.686331 10994 raft_consensus.cc:729] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dba36eaaf7a4fb39a4c85cff6f4a85b, State: Initialized, Role: FOLLOWER
I20260925 00:43:18.686811 10994 consensus_queue.cc:260] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 } } peers { permanent_uuid: "9e8ae180203c4941bac4c73dc34db4ae" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 42025 } } peers { permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 } }
I20260925 00:43:18.687412 10711 heartbeater.cc:503] Master 127.6.72.126:40005 was elected leader, sending a full tablet report...
I20260925 00:43:18.687801 10994 ts_tablet_manager.cc:1428] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 00:43:18.688441 10995 raft_consensus.cc:348] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 } } peers { permanent_uuid: "9e8ae180203c4941bac4c73dc34db4ae" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 42025 } } peers { permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 } }
I20260925 00:43:18.688719 10995 raft_consensus.cc:374] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:18.688803 10995 raft_consensus.cc:729] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a243504a46e45eb91d9831335354979, State: Initialized, Role: FOLLOWER
I20260925 00:43:18.689256 10995 consensus_queue.cc:260] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 } } peers { permanent_uuid: "9e8ae180203c4941bac4c73dc34db4ae" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 42025 } } peers { permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 } }
I20260925 00:43:18.689641 10996 raft_consensus.cc:348] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 } } peers { permanent_uuid: "9e8ae180203c4941bac4c73dc34db4ae" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 42025 } } peers { permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 } }
I20260925 00:43:18.689914 10996 raft_consensus.cc:374] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:18.689929 10844 heartbeater.cc:503] Master 127.6.72.126:40005 was elected leader, sending a full tablet report...
I20260925 00:43:18.689987 10996 raft_consensus.cc:729] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e8ae180203c4941bac4c73dc34db4ae, State: Initialized, Role: FOLLOWER
I20260925 00:43:18.690240 10995 ts_tablet_manager.cc:1428] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 00:43:18.690390 10996 consensus_queue.cc:260] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 } } peers { permanent_uuid: "9e8ae180203c4941bac4c73dc34db4ae" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 42025 } } peers { permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 } }
I20260925 00:43:18.691038 10977 heartbeater.cc:503] Master 127.6.72.126:40005 was elected leader, sending a full tablet report...
I20260925 00:43:18.691366 10996 ts_tablet_manager.cc:1428] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 00:43:18.806746 11002 raft_consensus.cc:482] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:18.806972 11002 raft_consensus.cc:504] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 } } peers { permanent_uuid: "9e8ae180203c4941bac4c73dc34db4ae" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 42025 } } peers { permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 } }
I20260925 00:43:18.808139 11002 leader_election.cc:288] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dba36eaaf7a4fb39a4c85cff6f4a85b (127.6.72.65:32861), 8a243504a46e45eb91d9831335354979 (127.6.72.66:34119)
I20260925 00:43:18.813397 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.808450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:34119 (local address 127.6.72.67:39427)
0925 00:43:18.808580 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:18.808594 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:18.808734 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:18.809149 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:18.809151 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:18.809164 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:18.809312 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.809320 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.810390 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.810394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.811899 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.811910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.812814 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.812818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.813009 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.813060 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 00:43:18.813154 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 00:43:18.813249 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":170}
I20260925 00:43:18.813396 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.808761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:39427 (local address 127.6.72.66:34119)
0925 00:43:18.808895 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:18.808898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:18.808912 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:18.808958 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:18.808960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:18.809034 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:18.809120 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:18.809436 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.810244 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.812033 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.812654 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.812767 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.812836 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:18.813185 (+ 349us) server_negotiation.cc:299] Negotiation successful
0925 00:43:18.813269 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:43:18.813431 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.808571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:56067 (local address 127.6.72.65:32861)
0925 00:43:18.808716 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:18.808721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:18.809020 (+ 299us) server_negotiation.cc:407] Connection header received
0925 00:43:18.809079 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:18.809083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:18.809134 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:18.809204 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:18.809436 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.810265 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.812033 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.812654 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.812778 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.812836 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:18.813231 (+ 395us) server_negotiation.cc:299] Negotiation successful
0925 00:43:18.813318 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 00:43:18.813803 11003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.808450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:32861 (local address 127.6.72.67:56067)
0925 00:43:18.808925 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:18.808935 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:18.809012 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:18.809231 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:18.809233 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:18.809244 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:18.809332 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.809338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.810390 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.810394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.811905 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:18.811910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.812814 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.812818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:18.812993 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.813055 (+ 62us) client_negotiation.cc:769] Sending connection context
0925 00:43:18.813550 (+ 495us) client_negotiation.cc:240] Negotiation successful
0925 00:43:18.813639 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":263,"thread_start_us":152,"threads_started":1}
I20260925 00:43:18.814457 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f7a881dbfe4cf49b211d2fbe3b38c6" candidate_uuid: "9e8ae180203c4941bac4c73dc34db4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a243504a46e45eb91d9831335354979" is_pre_election: true
I20260925 00:43:18.814500 10665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f7a881dbfe4cf49b211d2fbe3b38c6" candidate_uuid: "9e8ae180203c4941bac4c73dc34db4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" is_pre_election: true
I20260925 00:43:18.814792 10665 raft_consensus.cc:2454] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e8ae180203c4941bac4c73dc34db4ae in term 0.
I20260925 00:43:18.814795 10798 raft_consensus.cc:2454] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e8ae180203c4941bac4c73dc34db4ae in term 0.
I20260925 00:43:18.815308 10866 leader_election.cc:302] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0dba36eaaf7a4fb39a4c85cff6f4a85b, 8a243504a46e45eb91d9831335354979, 9e8ae180203c4941bac4c73dc34db4ae; no voters:
I20260925 00:43:18.815626 11002 raft_consensus.cc:2781] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:18.815711 11002 raft_consensus.cc:482] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:18.815754 11002 raft_consensus.cc:3037] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:18.816845 11002 raft_consensus.cc:504] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 } } peers { permanent_uuid: "9e8ae180203c4941bac4c73dc34db4ae" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 42025 } } peers { permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 } }
I20260925 00:43:18.817219 11002 leader_election.cc:288] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [CANDIDATE]: Term 1 election: Requested vote from peers 0dba36eaaf7a4fb39a4c85cff6f4a85b (127.6.72.65:32861), 8a243504a46e45eb91d9831335354979 (127.6.72.66:34119)
I20260925 00:43:18.817576 10665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f7a881dbfe4cf49b211d2fbe3b38c6" candidate_uuid: "9e8ae180203c4941bac4c73dc34db4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b"
I20260925 00:43:18.817605 10798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f7a881dbfe4cf49b211d2fbe3b38c6" candidate_uuid: "9e8ae180203c4941bac4c73dc34db4ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a243504a46e45eb91d9831335354979"
I20260925 00:43:18.817709 10798 raft_consensus.cc:3037] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:18.817710 10665 raft_consensus.cc:3037] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:18.819054 10665 raft_consensus.cc:2454] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e8ae180203c4941bac4c73dc34db4ae in term 1.
I20260925 00:43:18.819054 10798 raft_consensus.cc:2454] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e8ae180203c4941bac4c73dc34db4ae in term 1.
I20260925 00:43:18.819391 10865 leader_election.cc:302] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dba36eaaf7a4fb39a4c85cff6f4a85b, 9e8ae180203c4941bac4c73dc34db4ae; no voters:
I20260925 00:43:18.819589 11002 raft_consensus.cc:2781] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:18.819840 11002 raft_consensus.cc:686] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 1 LEADER]: Becoming Leader. State: Replica: 9e8ae180203c4941bac4c73dc34db4ae, State: Running, Role: LEADER
I20260925 00:43:18.820200 11002 consensus_queue.cc:237] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 } } peers { permanent_uuid: "9e8ae180203c4941bac4c73dc34db4ae" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 42025 } } peers { permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 } }
I20260925 00:43:18.823683 10521 catalog_manager.cc:5495] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e8ae180203c4941bac4c73dc34db4ae (127.6.72.67). New cstate: current_term: 1 leader_uuid: "9e8ae180203c4941bac4c73dc34db4ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e8ae180203c4941bac4c73dc34db4ae" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 42025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 } health_report { overall_health: UNKNOWN } } }
W20260925 00:43:18.853946 10978 tablet.cc:2369] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:43:18.859709 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.856357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.6.72.67:42025)
0925 00:43:18.856500 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:18.856504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:18.856518 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:18.856691 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:18.856693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:18.856747 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:18.856835 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:18.857045 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.857534 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.858321 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:18.858522 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:18.858601 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:18.858872 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:18.859352 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:18.859441 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:18.859491 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 00:43:18.859542 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 00:43:18.865832 10665 raft_consensus.cc:1261] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 0dba36eaaf7a4fb39a4c85cff6f4a85b [term 1 FOLLOWER]: Refusing update from remote peer 9e8ae180203c4941bac4c73dc34db4ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:18.865832 10798 raft_consensus.cc:1261] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979 [term 1 FOLLOWER]: Refusing update from remote peer 9e8ae180203c4941bac4c73dc34db4ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:18.866626 11004 consensus_queue.cc:1035] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dba36eaaf7a4fb39a4c85cff6f4a85b" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 32861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:18.866853 11002 consensus_queue.cc:1035] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a243504a46e45eb91d9831335354979" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 34119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:18.885425 10520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:18.885658 10520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:18.888501 10520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f697787f78974ef1ad6e7cd06e0d2cbd: Sending DeleteTablet for 3 replicas of tablet 55f7a881dbfe4cf49b211d2fbe3b38c6
I20260925 00:43:18.889439 10645 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f7a881dbfe4cf49b211d2fbe3b38c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:18 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:39498
I20260925 00:43:18.889730 10778 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f7a881dbfe4cf49b211d2fbe3b38c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:18 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:60784
I20260925 00:43:18.890133 10911 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f7a881dbfe4cf49b211d2fbe3b38c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:18 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:41600
I20260925 00:43:18.890668 11019 tablet_replica.cc:330] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: stopping tablet replica
I20260925 00:43:18.890793 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 10583
I20260925 00:43:18.891012 11019 raft_consensus.cc:2229] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:18.891390 11019 raft_consensus.cc:2258] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:18.892928 11019 ts_tablet_manager.cc:1905] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:18.895582 11019 ts_tablet_manager.cc:1918] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:18.895872 11018 tablet_replica.cc:330] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: stopping tablet replica
I20260925 00:43:18.896215 11018 raft_consensus.cc:2229] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:18.896365 11019 log.cc:1196] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/55f7a881dbfe4cf49b211d2fbe3b38c6
I20260925 00:43:18.896651 11018 raft_consensus.cc:2258] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:18.896811 11019 ts_tablet_manager.cc:1939] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 9e8ae180203c4941bac4c73dc34db4ae: Deleting consensus metadata
I20260925 00:43:18.898188 11018 ts_tablet_manager.cc:1905] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:18.898180 10506 catalog_manager.cc:4841] TS 9e8ae180203c4941bac4c73dc34db4ae (127.6.72.67:42025): tablet 55f7a881dbfe4cf49b211d2fbe3b38c6 (table test-table [id=66c6a0d090eb4586897d81e89b006ea1]) successfully deleted
I20260925 00:43:18.901021 11018 ts_tablet_manager.cc:1918] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:18.901155 11018 log.cc:1196] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/55f7a881dbfe4cf49b211d2fbe3b38c6
I20260925 00:43:18.901460 11018 ts_tablet_manager.cc:1939] T 55f7a881dbfe4cf49b211d2fbe3b38c6 P 8a243504a46e45eb91d9831335354979: Deleting consensus metadata
I20260925 00:43:18.902696 10507 catalog_manager.cc:4841] TS 8a243504a46e45eb91d9831335354979 (127.6.72.66:34119): tablet 55f7a881dbfe4cf49b211d2fbe3b38c6 (table test-table [id=66c6a0d090eb4586897d81e89b006ea1]) successfully deleted
W20260925 00:43:18.903910 10506 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:32861 (error 108)
I20260925 00:43:18.904702 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 10716
I20260925 00:43:18.905455 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:18.904782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:32861 (local address 127.0.0.1:39500)
0925 00:43:18.905309 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:18.905380 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426}
W20260925 00:43:18.905730 10506 catalog_manager.cc:4584] TS 0dba36eaaf7a4fb39a4c85cff6f4a85b (127.6.72.65:32861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55f7a881dbfe4cf49b211d2fbe3b38c6: Network error: Client connection negotiation failed: client connection to 127.6.72.65:32861: connect: Connection refused (error 111)
I20260925 00:43:18.912547 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 10849
I20260925 00:43:18.918022 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 10490
2026-09-25T00:43:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2375 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T00:43:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:18Z Disabled control of system clock
I20260925 00:43:18.949507 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:38171
--webserver_interface=127.6.72.126
--webserver_port=0
--builtin_ntp_servers=127.6.72.84:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:38171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 00:43:19.066737 11024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:19.067039 11024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:19.067080 11024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:19.071456 11024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:19.071544 11024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:19.071560 11024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:19.071578 11024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:19.071595 11024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:19.076823 11024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:38171
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:38171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:19.077970 11024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:19.079412 11024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:19.088508 11032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:19.088541 11030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:19.088514 11029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:19.088593 11024 server_base.cc:1034] running on GCE node
I20260925 00:43:19.090090 11024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:19.091233 11024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:19.092389 11024 hybrid_clock.cc:648] HybridClock initialized: now 1790296999092347 us; error 62 us; skew 500 ppm
I20260925 00:43:19.094429 11024 webserver.cc:460] Webserver started at http://127.6.72.126:38337/ using document root <none> and password file <none>
I20260925 00:43:19.094988 11024 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:19.095054 11024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:19.095234 11024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:19.097445 11024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "0e810ea7847749c7b5cad8de129a978b"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "41844b25db9720ae192a9eada15a092e"
server_key_iv: "ecbda5414e521f60ea4d1f5b347891d9"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.097980 11024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "0e810ea7847749c7b5cad8de129a978b"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "41844b25db9720ae192a9eada15a092e"
server_key_iv: "ecbda5414e521f60ea4d1f5b347891d9"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.101962 11024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:43:19.104584 11038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:19.105928 11024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:19.106072 11024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "0e810ea7847749c7b5cad8de129a978b"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "41844b25db9720ae192a9eada15a092e"
server_key_iv: "ecbda5414e521f60ea4d1f5b347891d9"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.106204 11024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:19.110983 11024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:19.111627 11024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:19.111827 11024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:19.118863 11024 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:38171
I20260925 00:43:19.118884 11090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:38171 every 8 connection(s)
I20260925 00:43:19.119839 11024 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:19.123368 11091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:19.126147 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 11024
I20260925 00:43:19.126307 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:19.126632 6433 external_mini_cluster.cc:1428] Setting key 6bae610ff1bd0a843300b4878b702304
I20260925 00:43:19.130542 11091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b: Bootstrap starting.
I20260925 00:43:19.133752 11091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:19.134840 11091 log.cc:824] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:19.136714 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.128505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.6.72.126:38171)
0925 00:43:19.131545 (+ 3040us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.131567 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.131626 (+ 59us) server_negotiation.cc:407] Connection header received
0925 00:43:19.132666 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.132693 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.133198 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.133632 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.134812 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.134818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.134835 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.134847 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.134921 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.134948 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.135067 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.135312 (+ 245us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.135601 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2563,"thread_start_us":146,"threads_started":1}
I20260925 00:43:19.137913 11091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b: No bootstrap required, opened a new log
I20260925 00:43:19.142096 11091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e810ea7847749c7b5cad8de129a978b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38171 } }
I20260925 00:43:19.142359 11091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:19.142396 11091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e810ea7847749c7b5cad8de129a978b, State: Initialized, Role: FOLLOWER
I20260925 00:43:19.142810 11091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e810ea7847749c7b5cad8de129a978b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38171 } }
I20260925 00:43:19.142932 11091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:19.143038 11091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:19.143142 11091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:19.144196 11091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e810ea7847749c7b5cad8de129a978b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38171 } }
I20260925 00:43:19.144654 11091 leader_election.cc:302] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e810ea7847749c7b5cad8de129a978b; no voters:
I20260925 00:43:19.145048 11091 leader_election.cc:288] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:19.145128 11096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:19.145403 11096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [term 1 LEADER]: Becoming Leader. State: Replica: 0e810ea7847749c7b5cad8de129a978b, State: Running, Role: LEADER
I20260925 00:43:19.145699 11096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e810ea7847749c7b5cad8de129a978b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38171 } }
I20260925 00:43:19.146446 11091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:19.147541 11097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e810ea7847749c7b5cad8de129a978b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e810ea7847749c7b5cad8de129a978b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38171 } } }
I20260925 00:43:19.147712 11097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:19.147527 11098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e810ea7847749c7b5cad8de129a978b. Latest consensus state: current_term: 1 leader_uuid: "0e810ea7847749c7b5cad8de129a978b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e810ea7847749c7b5cad8de129a978b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38171 } } }
I20260925 00:43:19.148082 11105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:19.148789 11098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:19.151799 11105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:19.157837 11105 catalog_manager.cc:1347] Generated new cluster ID: 9a765008635b48a9ba8ca44582cec78d
I20260925 00:43:19.157953 11105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:19.183383 11105 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:19.183761 11105 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:19.190250 11105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b: Generated new TSK 0
I20260925 00:43:19.191066 11105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:43:19.207926 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:38171
--builtin_ntp_servers=127.6.72.84:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 00:43:19.326934 11115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:19.327240 11115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:19.327283 11115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:19.331460 11115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:19.331539 11115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:19.331601 11115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:19.336678 11115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:38171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:19.337783 11115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:19.339263 11115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:19.349469 11123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:19.349478 11120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:19.349607 11115 server_base.cc:1034] running on GCE node
W20260925 00:43:19.349472 11121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:19.350310 11115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:19.351042 11115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:19.352283 11115 hybrid_clock.cc:648] HybridClock initialized: now 1790296999352225 us; error 30 us; skew 500 ppm
I20260925 00:43:19.354233 11115 webserver.cc:460] Webserver started at http://127.6.72.65:42549/ using document root <none> and password file <none>
I20260925 00:43:19.354899 11115 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:19.355000 11115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:19.355221 11115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:19.357466 11115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "2854dd17d74e4b71bec289a129776787"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "e8277f4dbe428727690aff611353ce26"
server_key_iv: "0c795b1eece38165bd9889df46f3d5e8"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.358058 11115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "2854dd17d74e4b71bec289a129776787"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "e8277f4dbe428727690aff611353ce26"
server_key_iv: "0c795b1eece38165bd9889df46f3d5e8"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.362012 11115 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:43:19.364638 11129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:19.365901 11115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 00:43:19.366045 11115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "2854dd17d74e4b71bec289a129776787"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "e8277f4dbe428727690aff611353ce26"
server_key_iv: "0c795b1eece38165bd9889df46f3d5e8"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.366175 11115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:19.370896 11115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:19.371538 11115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:19.371724 11115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:19.372342 11115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:19.373273 11115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:19.373342 11115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:19.373409 11115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:19.373456 11115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:19.381901 11115 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:42955
I20260925 00:43:19.381923 11242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:42955 every 8 connection(s)
I20260925 00:43:19.382822 11115 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:19.384909 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 11115
I20260925 00:43:19.385047 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:19.385314 6433 external_mini_cluster.cc:1428] Setting key c20d55679468ad0d4320d54b3979e40c
I20260925 00:43:19.387214 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:38171
--builtin_ntp_servers=127.6.72.84:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 00:43:19.389561 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.384886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:51143 (local address 127.6.72.126:38171)
0925 00:43:19.385063 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.385069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.386290 (+ 1221us) server_negotiation.cc:407] Connection header received
0925 00:43:19.387656 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.387662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.387711 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.387804 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.389124 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.389127 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.389129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.389140 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.389160 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.389168 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.389254 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.389421 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.389458 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 00:43:19.390553 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.385335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:38171 (local address 127.6.72.65:51143)
0925 00:43:19.386127 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:19.386180 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:19.387423 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:19.387966 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:19.387973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:19.388067 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:19.388770 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:19.388772 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:19.388850 (+ 78us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:19.388952 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:19.389257 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:19.389274 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:19.389280 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 00:43:19.389463 (+ 183us) client_negotiation.cc:240] Negotiation successful
0925 00:43:19.389741 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":153,"threads_started":1}
I20260925 00:43:19.391887 11243 heartbeater.cc:347] Connected to a master server at 127.6.72.126:38171
I20260925 00:43:19.392127 11243 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:19.392587 11243 heartbeater.cc:511] Master 127.6.72.126:38171 requested a full tablet report, sending...
I20260925 00:43:19.394204 11055 ts_manager.cc:194] Registered new tserver with Master: 2854dd17d74e4b71bec289a129776787 (127.6.72.65:42955)
W20260925 00:43:19.501648 11246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:19.501979 11246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:19.502055 11246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:19.506304 11246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:19.506402 11246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:19.506500 11246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:19.511533 11246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:38171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:19.512699 11246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:19.514221 11246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:19.524025 11251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:19.524031 11254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:19.524185 11252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:19.524500 11246 server_base.cc:1034] running on GCE node
I20260925 00:43:19.524923 11246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:19.525576 11246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:19.526760 11246 hybrid_clock.cc:648] HybridClock initialized: now 1790296999526725 us; error 50 us; skew 500 ppm
I20260925 00:43:19.528815 11246 webserver.cc:460] Webserver started at http://127.6.72.66:40979/ using document root <none> and password file <none>
I20260925 00:43:19.529378 11246 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:19.529438 11246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:19.529603 11246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:19.531636 11246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "e212ac6b8fd74da0bed072e96034ea88"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "f9bdb46ea2b722b08ff164f259830e15"
server_key_iv: "3db428adc367e7b8e09ecd1ead110c70"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.532140 11246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "e212ac6b8fd74da0bed072e96034ea88"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "f9bdb46ea2b722b08ff164f259830e15"
server_key_iv: "3db428adc367e7b8e09ecd1ead110c70"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.536154 11246 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 00:43:19.538815 11260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:19.540320 11246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 00:43:19.540500 11246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "e212ac6b8fd74da0bed072e96034ea88"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "f9bdb46ea2b722b08ff164f259830e15"
server_key_iv: "3db428adc367e7b8e09ecd1ead110c70"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.540628 11246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:19.545631 11246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:19.546370 11246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:19.546564 11246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:19.547322 11246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:19.548646 11246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:19.548712 11246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:19.548772 11246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:19.548802 11246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:19.559890 11246 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:39577
I20260925 00:43:19.559908 11373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:39577 every 8 connection(s)
I20260925 00:43:19.560956 11246 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:19.564484 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 11246
I20260925 00:43:19.564634 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:19.564918 6433 external_mini_cluster.cc:1428] Setting key d3979e44889d089aa5db4ed873a9243f
I20260925 00:43:19.566898 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:38171
--builtin_ntp_servers=127.6.72.84:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 00:43:19.569519 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.562916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:53673 (local address 127.6.72.126:38171)
0925 00:43:19.563054 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.563057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.564083 (+ 1026us) server_negotiation.cc:407] Connection header received
0925 00:43:19.565636 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.565640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.565685 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.565767 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.568989 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.568992 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.568994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.569010 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.569032 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.569042 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.569128 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.569357 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.569403 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 00:43:19.570858 11376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.563279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:38171 (local address 127.6.72.66:53673)
0925 00:43:19.563933 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:19.563975 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:19.565414 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:19.565953 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:19.565960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:19.566052 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:19.568558 (+ 2506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:19.568562 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:19.568645 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:19.568772 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:19.569147 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:19.569160 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:19.569193 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 00:43:19.569406 (+ 213us) client_negotiation.cc:240] Negotiation successful
0925 00:43:19.569716 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":109,"threads_started":1}
I20260925 00:43:19.572325 11374 heartbeater.cc:347] Connected to a master server at 127.6.72.126:38171
I20260925 00:43:19.572587 11374 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:19.573038 11374 heartbeater.cc:511] Master 127.6.72.126:38171 requested a full tablet report, sending...
I20260925 00:43:19.574232 11055 ts_manager.cc:194] Registered new tserver with Master: e212ac6b8fd74da0bed072e96034ea88 (127.6.72.66:39577)
W20260925 00:43:19.682854 11377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:19.683187 11377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:19.683262 11377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:19.687623 11377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:19.687718 11377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:19.687809 11377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:19.692883 11377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:38171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:19.694015 11377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:19.695511 11377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:19.705271 11385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:19.705318 11377 server_base.cc:1034] running on GCE node
W20260925 00:43:19.705263 11382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:19.705255 11383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:19.705830 11377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:19.706521 11377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:19.707742 11377 hybrid_clock.cc:648] HybridClock initialized: now 1790296999707708 us; error 40 us; skew 500 ppm
I20260925 00:43:19.709830 11377 webserver.cc:460] Webserver started at http://127.6.72.67:39373/ using document root <none> and password file <none>
I20260925 00:43:19.710465 11377 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:19.710554 11377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:19.710805 11377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:19.712949 11377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "440876c2bece4f3d90603b3588c5522c"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "d5bb09e4a8c778cd189f8dbccf05831a"
server_key_iv: "faed7a4cf600b87f357265b6a5af37b1"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.713558 11377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "440876c2bece4f3d90603b3588c5522c"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "d5bb09e4a8c778cd189f8dbccf05831a"
server_key_iv: "faed7a4cf600b87f357265b6a5af37b1"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.717278 11377 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 00:43:19.719673 11391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:19.721156 11377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:19.721290 11377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "440876c2bece4f3d90603b3588c5522c"
format_stamp: "Formatted at 2026-09-25 00:43:19 on dist-test-slave-w8nh"
server_key: "d5bb09e4a8c778cd189f8dbccf05831a"
server_key_iv: "faed7a4cf600b87f357265b6a5af37b1"
server_key_version: "encryptionkey@0"
I20260925 00:43:19.721410 11377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:19.726135 11377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:19.726760 11377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:19.726943 11377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:19.727500 11377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:19.728487 11377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:19.728554 11377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:19.728622 11377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:19.728673 11377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:19.737326 11377 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:40887
I20260925 00:43:19.737347 11504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:40887 every 8 connection(s)
I20260925 00:43:19.738284 11377 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:19.744143 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 11377
I20260925 00:43:19.744315 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:19.744452 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.740162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:47027 (local address 127.6.72.126:38171)
0925 00:43:19.740345 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.740350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.741428 (+ 1078us) server_negotiation.cc:407] Connection header received
0925 00:43:19.742647 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.742649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.742687 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.742761 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.743877 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.743883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.743890 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.743903 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.743940 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.743949 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.744029 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.744197 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.744294 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 00:43:19.744619 6433 external_mini_cluster.cc:1428] Setting key ff9123ce82ed52e732b5a796e52fa930
I20260925 00:43:19.745507 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.740612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:38171 (local address 127.6.72.67:47027)
0925 00:43:19.741287 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:19.741324 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:19.742471 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:19.742904 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:19.742910 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:19.742995 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:19.743510 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:19.743511 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:19.743550 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:19.743614 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:19.744040 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:19.744049 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:19.744054 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 00:43:19.744296 (+ 242us) client_negotiation.cc:240] Negotiation successful
0925 00:43:19.744598 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":129,"threads_started":1}
I20260925 00:43:19.746803 11505 heartbeater.cc:347] Connected to a master server at 127.6.72.126:38171
I20260925 00:43:19.747033 11505 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:19.747452 11505 heartbeater.cc:511] Master 127.6.72.126:38171 requested a full tablet report, sending...
I20260925 00:43:19.748487 11055 ts_manager.cc:194] Registered new tserver with Master: 440876c2bece4f3d90603b3588c5522c (127.6.72.67:40887)
I20260925 00:43:19.758527 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:43:19.761907 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.760351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53860 (local address 127.6.72.126:38171)
0925 00:43:19.760512 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.760517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.760748 (+ 231us) server_negotiation.cc:407] Connection header received
0925 00:43:19.760845 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.760847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.760882 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.760945 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.761562 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.761565 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.761567 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.761578 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.761594 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.761602 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.761662 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.761706 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.761738 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 00:43:19.764628 11055 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:19.767555 11055 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:19.779772 11517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.776238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36326 (local address 127.6.72.65:42955)
0925 00:43:19.777510 (+ 1272us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.777522 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.777548 (+ 26us) server_negotiation.cc:407] Connection header received
0925 00:43:19.777606 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.777610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.777761 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.777926 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.778882 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.778891 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.778904 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.778917 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.778959 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.778992 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.779092 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.779520 (+ 428us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.779599 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1162,"thread_start_us":104,"threads_started":1}
I20260925 00:43:19.779790 11518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.776149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:42955 (local address 127.0.0.1:36326)
0925 00:43:19.776993 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:19.777046 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:19.777146 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:19.777924 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:19.777930 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:19.778004 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:19.778629 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:19.778631 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:19.778652 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:19.778726 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:19.779136 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:19.779144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:19.779149 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 00:43:19.779291 (+ 142us) client_negotiation.cc:240] Negotiation successful
0925 00:43:19.779367 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":124,"threads_started":1}
I20260925 00:43:19.780166 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.776230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:40887 (local address 127.0.0.1:37496)
0925 00:43:19.777101 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:19.777113 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:19.777169 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:19.778668 (+ 1499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:19.778670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:19.778684 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:19.779352 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:19.779354 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:19.779364 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:19.779390 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:19.779906 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:19.779910 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:19.779914 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 00:43:19.780006 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 00:43:19.780044 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":110,"threads_started":1}
I20260925 00:43:19.780299 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.776813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37496 (local address 127.6.72.67:40887)
0925 00:43:19.777688 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.777696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.777713 (+ 17us) server_negotiation.cc:407] Connection header received
0925 00:43:19.777762 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.777766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.777949 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.778085 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.779661 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.779667 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.779678 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.779689 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.779728 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.779768 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.779865 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.780048 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.780123 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":149,"threads_started":1}
I20260925 00:43:19.781306 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.776858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:39577 (local address 127.0.0.1:50174)
0925 00:43:19.778666 (+ 1808us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:19.778679 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:19.778760 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:19.779153 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:19.779156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:19.779169 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:19.779520 (+ 351us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:19.779522 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:19.779530 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:19.779548 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:19.781079 (+ 1531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:19.781082 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:19.781085 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 00:43:19.781161 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 00:43:19.781195 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1747,"thread_start_us":97,"threads_started":1}
I20260925 00:43:19.781569 11521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.777298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50174 (local address 127.6.72.66:39577)
0925 00:43:19.777580 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.777592 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.778785 (+ 1193us) server_negotiation.cc:407] Connection header received
0925 00:43:19.778850 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.778858 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.779016 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.779166 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.779726 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.779730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.780908 (+ 1178us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.780924 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.780966 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.780999 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.781105 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.781175 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.781238 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":102,"threads_started":1}
I20260925 00:43:19.782989 11439 tablet_service.cc:1467] Processing CreateTablet for tablet 18171a5a9dcd47b297592c6fa65aebc3 (DEFAULT_TABLE table=test-table [id=72406116d7f64c16bc084e76fe8acf33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:19.782989 11177 tablet_service.cc:1467] Processing CreateTablet for tablet 18171a5a9dcd47b297592c6fa65aebc3 (DEFAULT_TABLE table=test-table [id=72406116d7f64c16bc084e76fe8acf33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:19.784107 11177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18171a5a9dcd47b297592c6fa65aebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:19.784107 11439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18171a5a9dcd47b297592c6fa65aebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:19.783888 11308 tablet_service.cc:1467] Processing CreateTablet for tablet 18171a5a9dcd47b297592c6fa65aebc3 (DEFAULT_TABLE table=test-table [id=72406116d7f64c16bc084e76fe8acf33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:19.784891 11308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18171a5a9dcd47b297592c6fa65aebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:19.790934 11522 tablet_bootstrap.cc:492] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88: Bootstrap starting.
I20260925 00:43:19.792484 11523 tablet_bootstrap.cc:492] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: Bootstrap starting.
I20260925 00:43:19.793876 11522 tablet_bootstrap.cc:654] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:19.794572 11523 tablet_bootstrap.cc:654] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:19.794790 11522 log.cc:824] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:19.795408 11523 log.cc:824] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:19.795538 11524 tablet_bootstrap.cc:492] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787: Bootstrap starting.
I20260925 00:43:19.797299 11522 tablet_bootstrap.cc:492] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88: No bootstrap required, opened a new log
I20260925 00:43:19.797400 11523 tablet_bootstrap.cc:492] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: No bootstrap required, opened a new log
I20260925 00:43:19.797544 11522 ts_tablet_manager.cc:1397] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 00:43:19.797585 11523 ts_tablet_manager.cc:1397] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 00:43:19.797610 11524 tablet_bootstrap.cc:654] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:19.798381 11524 log.cc:824] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:19.800400 11524 tablet_bootstrap.cc:492] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787: No bootstrap required, opened a new log
I20260925 00:43:19.800619 11524 ts_tablet_manager.cc:1397] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260925 00:43:19.801605 11522 raft_consensus.cc:348] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 } } peers { permanent_uuid: "440876c2bece4f3d90603b3588c5522c" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 40887 } } peers { permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 } }
I20260925 00:43:19.801846 11522 raft_consensus.cc:374] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:19.801904 11522 raft_consensus.cc:729] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e212ac6b8fd74da0bed072e96034ea88, State: Initialized, Role: FOLLOWER
I20260925 00:43:19.801954 11523 raft_consensus.cc:348] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 } } peers { permanent_uuid: "440876c2bece4f3d90603b3588c5522c" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 40887 } } peers { permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 } }
I20260925 00:43:19.802217 11523 raft_consensus.cc:374] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:19.802268 11523 raft_consensus.cc:729] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 440876c2bece4f3d90603b3588c5522c, State: Initialized, Role: FOLLOWER
I20260925 00:43:19.802408 11522 consensus_queue.cc:260] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 } } peers { permanent_uuid: "440876c2bece4f3d90603b3588c5522c" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 40887 } } peers { permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 } }
I20260925 00:43:19.802698 11523 consensus_queue.cc:260] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 } } peers { permanent_uuid: "440876c2bece4f3d90603b3588c5522c" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 40887 } } peers { permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 } }
I20260925 00:43:19.803154 11374 heartbeater.cc:503] Master 127.6.72.126:38171 was elected leader, sending a full tablet report...
I20260925 00:43:19.803750 11523 ts_tablet_manager.cc:1428] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 00:43:19.804096 11505 heartbeater.cc:503] Master 127.6.72.126:38171 was elected leader, sending a full tablet report...
I20260925 00:43:19.804834 11522 ts_tablet_manager.cc:1428] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260925 00:43:19.805116 11524 raft_consensus.cc:348] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 } } peers { permanent_uuid: "440876c2bece4f3d90603b3588c5522c" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 40887 } } peers { permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 } }
I20260925 00:43:19.805349 11524 raft_consensus.cc:374] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:19.805400 11524 raft_consensus.cc:729] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2854dd17d74e4b71bec289a129776787, State: Initialized, Role: FOLLOWER
I20260925 00:43:19.805876 11524 consensus_queue.cc:260] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 } } peers { permanent_uuid: "440876c2bece4f3d90603b3588c5522c" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 40887 } } peers { permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 } }
I20260925 00:43:19.806550 11243 heartbeater.cc:503] Master 127.6.72.126:38171 was elected leader, sending a full tablet report...
I20260925 00:43:19.806928 11524 ts_tablet_manager.cc:1428] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260925 00:43:19.811882 11375 tablet.cc:2369] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:43:19.839118 11529 raft_consensus.cc:482] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:19.839354 11529 raft_consensus.cc:504] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 } } peers { permanent_uuid: "440876c2bece4f3d90603b3588c5522c" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 40887 } } peers { permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 } }
I20260925 00:43:19.840572 11529 leader_election.cc:288] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2854dd17d74e4b71bec289a129776787 (127.6.72.65:42955), e212ac6b8fd74da0bed072e96034ea88 (127.6.72.66:39577)
I20260925 00:43:19.842483 11517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.840913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:44063 (local address 127.6.72.65:42955)
0925 00:43:19.841039 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.841043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.841057 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:19.841191 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.841195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.841245 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.841316 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.842039 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.842063 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.842067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.842083 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.842105 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.842115 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.842212 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.842309 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.842354 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 00:43:19.842482 11521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.841029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:36479 (local address 127.6.72.66:39577)
0925 00:43:19.841186 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.841190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.841397 (+ 207us) server_negotiation.cc:407] Connection header received
0925 00:43:19.841461 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.841463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.841494 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.841590 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.842120 (+ 530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.842123 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.842126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.842142 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.842164 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.842177 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.842261 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.842334 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.842367 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 00:43:19.842491 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.840776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:42955 (local address 127.6.72.67:44063)
0925 00:43:19.840972 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:19.840989 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:19.841071 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:19.841373 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:19.841376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:19.841388 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:19.841858 (+ 470us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:19.841859 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:19.841868 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:19.841913 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:19.842201 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:19.842203 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:19.842206 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 00:43:19.842311 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 00:43:19.842350 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260925 00:43:19.842507 11531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.840905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:39577 (local address 127.6.72.67:36479)
0925 00:43:19.841267 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:19.841279 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:19.841386 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:19.841602 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:19.841609 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:19.841621 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:19.841968 (+ 347us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:19.841971 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:19.841982 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:19.842006 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:19.842255 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:19.842259 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:19.842265 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 00:43:19.842335 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 00:43:19.842357 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":132,"threads_started":1}
I20260925 00:43:19.843194 11197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18171a5a9dcd47b297592c6fa65aebc3" candidate_uuid: "440876c2bece4f3d90603b3588c5522c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2854dd17d74e4b71bec289a129776787" is_pre_election: true
I20260925 00:43:19.843271 11328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18171a5a9dcd47b297592c6fa65aebc3" candidate_uuid: "440876c2bece4f3d90603b3588c5522c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e212ac6b8fd74da0bed072e96034ea88" is_pre_election: true
I20260925 00:43:19.843518 11328 raft_consensus.cc:2454] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 440876c2bece4f3d90603b3588c5522c in term 0.
I20260925 00:43:19.843518 11197 raft_consensus.cc:2454] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 440876c2bece4f3d90603b3588c5522c in term 0.
I20260925 00:43:19.843959 11392 leader_election.cc:302] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 440876c2bece4f3d90603b3588c5522c, e212ac6b8fd74da0bed072e96034ea88; no voters:
I20260925 00:43:19.844298 11529 raft_consensus.cc:2781] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:19.844378 11529 raft_consensus.cc:482] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:19.844427 11529 raft_consensus.cc:3037] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:19.845495 11529 raft_consensus.cc:504] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 } } peers { permanent_uuid: "440876c2bece4f3d90603b3588c5522c" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 40887 } } peers { permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 } }
I20260925 00:43:19.845814 11529 leader_election.cc:288] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [CANDIDATE]: Term 1 election: Requested vote from peers 2854dd17d74e4b71bec289a129776787 (127.6.72.65:42955), e212ac6b8fd74da0bed072e96034ea88 (127.6.72.66:39577)
I20260925 00:43:19.846133 11197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18171a5a9dcd47b297592c6fa65aebc3" candidate_uuid: "440876c2bece4f3d90603b3588c5522c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2854dd17d74e4b71bec289a129776787"
I20260925 00:43:19.846133 11328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18171a5a9dcd47b297592c6fa65aebc3" candidate_uuid: "440876c2bece4f3d90603b3588c5522c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e212ac6b8fd74da0bed072e96034ea88"
I20260925 00:43:19.846264 11328 raft_consensus.cc:3037] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:19.846254 11197 raft_consensus.cc:3037] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:19.847522 11197 raft_consensus.cc:2454] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 440876c2bece4f3d90603b3588c5522c in term 1.
I20260925 00:43:19.847522 11328 raft_consensus.cc:2454] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 440876c2bece4f3d90603b3588c5522c in term 1.
I20260925 00:43:19.847857 11392 leader_election.cc:302] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 440876c2bece4f3d90603b3588c5522c, e212ac6b8fd74da0bed072e96034ea88; no voters:
I20260925 00:43:19.848019 11529 raft_consensus.cc:2781] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:19.848222 11529 raft_consensus.cc:686] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 1 LEADER]: Becoming Leader. State: Replica: 440876c2bece4f3d90603b3588c5522c, State: Running, Role: LEADER
I20260925 00:43:19.848587 11529 consensus_queue.cc:237] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 } } peers { permanent_uuid: "440876c2bece4f3d90603b3588c5522c" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 40887 } } peers { permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 } }
I20260925 00:43:19.851821 11055 catalog_manager.cc:5495] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c reported cstate change: term changed from 0 to 1, leader changed from <none> to 440876c2bece4f3d90603b3588c5522c (127.6.72.67). New cstate: current_term: 1 leader_uuid: "440876c2bece4f3d90603b3588c5522c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "440876c2bece4f3d90603b3588c5522c" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 40887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20260925 00:43:19.878912 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.877488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.6.72.67:40887)
0925 00:43:19.877627 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:19.877630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:19.877643 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:43:19.877714 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:19.877716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:19.877781 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:19.877850 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:19.878615 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:19.878618 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:19.878620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:19.878630 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:19.878649 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:19.878656 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:19.878729 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:19.878777 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 00:43:19.878809 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260925 00:43:19.884197 11244 tablet.cc:2369] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:43:19.884869 11328 raft_consensus.cc:1261] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88 [term 1 FOLLOWER]: Refusing update from remote peer 440876c2bece4f3d90603b3588c5522c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:19.884876 11197 raft_consensus.cc:1261] T 18171a5a9dcd47b297592c6fa65aebc3 P 2854dd17d74e4b71bec289a129776787 [term 1 FOLLOWER]: Refusing update from remote peer 440876c2bece4f3d90603b3588c5522c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:19.885542 11532 consensus_queue.cc:1035] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2854dd17d74e4b71bec289a129776787" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 42955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:19.885754 11529 consensus_queue.cc:1035] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e212ac6b8fd74da0bed072e96034ea88" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:19.902410 11055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:19.902591 11055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:19.905025 11055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e810ea7847749c7b5cad8de129a978b: Sending DeleteTablet for 3 replicas of tablet 18171a5a9dcd47b297592c6fa65aebc3
I20260925 00:43:19.905807 11439 tablet_service.cc:1514] Processing DeleteTablet for tablet 18171a5a9dcd47b297592c6fa65aebc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:19 UTC) from {username='slave'} at 127.0.0.1:37496
I20260925 00:43:19.905895 11177 tablet_service.cc:1514] Processing DeleteTablet for tablet 18171a5a9dcd47b297592c6fa65aebc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:19 UTC) from {username='slave'} at 127.0.0.1:36326
I20260925 00:43:19.906457 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 11115
I20260925 00:43:19.906520 11308 tablet_service.cc:1514] Processing DeleteTablet for tablet 18171a5a9dcd47b297592c6fa65aebc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:19 UTC) from {username='slave'} at 127.0.0.1:50174
I20260925 00:43:19.907811 11544 tablet_replica.cc:330] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: stopping tablet replica
I20260925 00:43:19.908208 11544 raft_consensus.cc:2229] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:19.908630 11544 raft_consensus.cc:2258] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:19.910072 11544 ts_tablet_manager.cc:1905] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:19.912452 11544 ts_tablet_manager.cc:1918] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:19.912551 11544 log.cc:1196] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/18171a5a9dcd47b297592c6fa65aebc3
I20260925 00:43:19.912874 11544 ts_tablet_manager.cc:1939] T 18171a5a9dcd47b297592c6fa65aebc3 P 440876c2bece4f3d90603b3588c5522c: Deleting consensus metadata
W20260925 00:43:19.913594 11040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:42955 (error 108)
I20260925 00:43:19.914084 11042 catalog_manager.cc:4841] TS 440876c2bece4f3d90603b3588c5522c (127.6.72.67:40887): tablet 18171a5a9dcd47b297592c6fa65aebc3 (table test-table [id=72406116d7f64c16bc084e76fe8acf33]) successfully deleted
I20260925 00:43:19.914141 11546 tablet_replica.cc:330] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88: stopping tablet replica
I20260925 00:43:19.914350 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 11246
I20260925 00:43:19.914490 11546 raft_consensus.cc:2229] T 18171a5a9dcd47b297592c6fa65aebc3 P e212ac6b8fd74da0bed072e96034ea88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:19.917074 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.916794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:42955 (local address 127.0.0.1:36336)
0925 00:43:19.916928 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:19.917000 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:42955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 00:43:19.917268 11040 catalog_manager.cc:4584] TS 2854dd17d74e4b71bec289a129776787 (127.6.72.65:42955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 18171a5a9dcd47b297592c6fa65aebc3: Network error: Client connection negotiation failed: client connection to 127.6.72.65:42955: connect: Connection refused (error 111)
I20260925 00:43:19.923576 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 11377
I20260925 00:43:19.925339 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:19.925074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:39577 (local address 127.0.0.1:50182)
0925 00:43:19.925204 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:19.925265 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.66:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 00:43:19.929271 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 11024
2026-09-25T00:43:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1009 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 00:43:19.951009 6433 mini_oidc.cc:152] Starting JWKS server
I20260925 00:43:19.952736 6433 webserver.cc:460] Webserver started at https://127.0.0.1:38875/ using document root <none> and password file <none>
I20260925 00:43:19.952935 6433 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 00:43:19.953224 6433 webserver.cc:460] Webserver started at http://127.0.0.1:42267/ using document root <none> and password file <none>
2026-09-25T00:43:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:19Z Disabled control of system clock
I20260925 00:43:19.962441 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:39365
--webserver_interface=127.6.72.126
--webserver_port=0
--builtin_ntp_servers=127.6.72.84:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:39365
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38875/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/testchainca.cert with env {}
W20260925 00:43:20.076042 11553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:20.076429 11553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:20.076478 11553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:20.080668 11553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:20.080744 11553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:20.080760 11553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:20.080777 11553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:20.080814 11553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 00:43:20.080832 11553 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 00:43:20.080845 11553 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38875/jwks.json
I20260925 00:43:20.086447 11553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:39365
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:39365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38875/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:20.087530 11553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:20.088528 11553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:20.096808 11558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:20.096917 11553 server_base.cc:1034] running on GCE node
W20260925 00:43:20.096833 11559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:20.096868 11561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:20.097661 11553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:20.098868 11553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:20.100060 11553 hybrid_clock.cc:648] HybridClock initialized: now 1790297000100028 us; error 49 us; skew 500 ppm
I20260925 00:43:20.102084 11553 webserver.cc:460] Webserver started at http://127.6.72.126:42701/ using document root <none> and password file <none>
I20260925 00:43:20.102730 11553 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:20.102828 11553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:20.103051 11553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:20.105227 11553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "19dd3b0dae9e4b77b19f85aa66ff893c"
format_stamp: "Formatted at 2026-09-25 00:43:20 on dist-test-slave-w8nh"
server_key: "8d1c0d3bb970d8de75970b5e5d49b840"
server_key_iv: "403b22f400547bddc8d2bf5bc8ae0ea5"
server_key_version: "encryptionkey@0"
I20260925 00:43:20.105818 11553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "19dd3b0dae9e4b77b19f85aa66ff893c"
format_stamp: "Formatted at 2026-09-25 00:43:20 on dist-test-slave-w8nh"
server_key: "8d1c0d3bb970d8de75970b5e5d49b840"
server_key_iv: "403b22f400547bddc8d2bf5bc8ae0ea5"
server_key_version: "encryptionkey@0"
I20260925 00:43:20.109468 11553 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:43:20.111948 11567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:20.113206 11553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 00:43:20.113346 11553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "19dd3b0dae9e4b77b19f85aa66ff893c"
format_stamp: "Formatted at 2026-09-25 00:43:20 on dist-test-slave-w8nh"
server_key: "8d1c0d3bb970d8de75970b5e5d49b840"
server_key_iv: "403b22f400547bddc8d2bf5bc8ae0ea5"
server_key_version: "encryptionkey@0"
I20260925 00:43:20.113462 11553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:20.136775 11553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:20.137514 11553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:20.137722 11553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:20.145087 11553 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:39365
I20260925 00:43:20.145100 11619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:39365 every 8 connection(s)
I20260925 00:43:20.146077 11553 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:20.148506 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 11553
I20260925 00:43:20.148660 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:20.148941 6433 external_mini_cluster.cc:1428] Setting key a7362711935af2f45fbd21747763926a
I20260925 00:43:20.149469 11620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:20.156487 11620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c: Bootstrap starting.
I20260925 00:43:20.158226 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:20.150255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.6.72.126:39365)
0925 00:43:20.150862 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:20.150878 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:20.150913 (+ 35us) server_negotiation.cc:407] Connection header received
0925 00:43:20.151556 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:20.151572 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:20.151965 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:20.152287 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:20.152663 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.153721 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.154926 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.155418 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.155535 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:20.156410 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:20.156414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:20.156427 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:20.156893 (+ 466us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:20.156948 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:20.156968 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:20.157043 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:20.157206 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 00:43:20.157493 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":133,"threads_started":1}
I20260925 00:43:20.158871 11620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:20.159586 11620 log.cc:824] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:20.161015 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:43:20.161739 11620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c: No bootstrap required, opened a new log
I20260925 00:43:20.165390 11620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19dd3b0dae9e4b77b19f85aa66ff893c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39365 } }
I20260925 00:43:20.165635 11620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:20.165669 11620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19dd3b0dae9e4b77b19f85aa66ff893c, State: Initialized, Role: FOLLOWER
I20260925 00:43:20.166051 11620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19dd3b0dae9e4b77b19f85aa66ff893c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39365 } }
I20260925 00:43:20.166158 11620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:20.166204 11620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:20.166276 11620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:20.167264 11620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19dd3b0dae9e4b77b19f85aa66ff893c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39365 } }
I20260925 00:43:20.167572 11620 leader_election.cc:302] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19dd3b0dae9e4b77b19f85aa66ff893c; no voters:
I20260925 00:43:20.167843 11620 leader_election.cc:288] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:20.167995 11625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:20.168277 11625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [term 1 LEADER]: Becoming Leader. State: Replica: 19dd3b0dae9e4b77b19f85aa66ff893c, State: Running, Role: LEADER
I20260925 00:43:20.168596 11625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19dd3b0dae9e4b77b19f85aa66ff893c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39365 } }
I20260925 00:43:20.169191 11620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:20.170387 11627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19dd3b0dae9e4b77b19f85aa66ff893c. Latest consensus state: current_term: 1 leader_uuid: "19dd3b0dae9e4b77b19f85aa66ff893c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19dd3b0dae9e4b77b19f85aa66ff893c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39365 } } }
I20260925 00:43:20.170562 11627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:20.170387 11626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19dd3b0dae9e4b77b19f85aa66ff893c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19dd3b0dae9e4b77b19f85aa66ff893c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39365 } } }
I20260925 00:43:20.170658 11626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:20.171456 11635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:20.174183 11635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:20.180112 11635 catalog_manager.cc:1347] Generated new cluster ID: 683109dbe7664757873f1c6580defc08
I20260925 00:43:20.180194 11635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:20.191143 11635 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:20.192121 11635 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:20.210681 11635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19dd3b0dae9e4b77b19f85aa66ff893c: Generated new TSK 0
I20260925 00:43:20.211481 11635 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 00:43:20.241351 11650 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 00:43:20.241477 11650 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 00:43:20.241991 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:20.224505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41914 (local address 127.6.72.126:39365)
0925 00:43:20.224651 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:20.224657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:20.224859 (+ 202us) server_negotiation.cc:407] Connection header received
0925 00:43:20.225060 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:20.225064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:20.225126 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:20.225245 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0925 00:43:20.225475 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.226317 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.227161 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.227526 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.227630 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:20.227966 (+ 336us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 00:43:20.241548 (+ 13582us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 00:43:20.241848 (+ 300us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41914: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 00:43:20.242079 11623 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41914: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260925 00:43:20.243175 11649 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:20.224439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:39365 (local address 127.0.0.1:41914)
0925 00:43:20.224774 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:20.224787 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:20.224899 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:20.225261 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:20.225263 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:20.225270 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0925 00:43:20.225372 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:20.225376 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.226427 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.226429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:20.227046 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:20.227052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.227640 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.227643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:20.227708 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:20.227763 (+ 55us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 00:43:20.241792 (+ 14029us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 00:43:20.242000 (+ 208us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.72.126:39365: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":119,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.72.126:39365: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 00:43:20.244211 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 11553
2026-09-25T00:43:20Z chronyd exiting
I20260925 00:43:20.281749 6433 test_util.cc:175] -----------------------------------------------
I20260925 00:43:20.281864 6433 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790296971355272-6433-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (332 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 00:43:20.282946 6433 mini_oidc.cc:152] Starting JWKS server
I20260925 00:43:20.283833 6433 webserver.cc:460] Webserver started at https://127.0.0.1:40411/ using document root <none> and password file <none>
I20260925 00:43:20.283963 6433 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 00:43:20.284276 6433 webserver.cc:460] Webserver started at http://127.0.0.1:35797/ using document root <none> and password file <none>
2026-09-25T00:43:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:20Z Disabled control of system clock
I20260925 00:43:20.293073 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:39771
--webserver_interface=127.6.72.126
--webserver_port=0
--builtin_ntp_servers=127.6.72.84:32905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:39771
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40411/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/testchainca.cert with env {}
W20260925 00:43:20.402125 11656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:20.402420 11656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:20.402462 11656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:20.406435 11656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:20.406493 11656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:20.406508 11656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:20.406525 11656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:20.406540 11656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 00:43:20.406556 11656 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 00:43:20.406570 11656 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40411/jwks.json
I20260925 00:43:20.411810 11656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:32905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:39771
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:39771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40411/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:20.412851 11656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:20.413798 11656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:20.422156 11662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:20.422166 11664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:20.422184 11661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:20.422489 11656 server_base.cc:1034] running on GCE node
I20260925 00:43:20.423022 11656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:20.424160 11656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:20.425347 11656 hybrid_clock.cc:648] HybridClock initialized: now 1790297000425339 us; error 28 us; skew 500 ppm
I20260925 00:43:20.427261 11656 webserver.cc:460] Webserver started at http://127.6.72.126:37035/ using document root <none> and password file <none>
I20260925 00:43:20.427861 11656 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:20.427966 11656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:20.428159 11656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:20.430210 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "6faf1788d7334495b1038793424499d6"
format_stamp: "Formatted at 2026-09-25 00:43:20 on dist-test-slave-w8nh"
server_key: "aa837d14f147932d6a09671b246553f3"
server_key_iv: "4e8a2c03e14d3ff0163126389f0a9609"
server_key_version: "encryptionkey@0"
I20260925 00:43:20.430721 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "6faf1788d7334495b1038793424499d6"
format_stamp: "Formatted at 2026-09-25 00:43:20 on dist-test-slave-w8nh"
server_key: "aa837d14f147932d6a09671b246553f3"
server_key_iv: "4e8a2c03e14d3ff0163126389f0a9609"
server_key_version: "encryptionkey@0"
I20260925 00:43:20.434320 11656 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 00:43:20.436724 11670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:20.438004 11656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:20.438160 11656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "6faf1788d7334495b1038793424499d6"
format_stamp: "Formatted at 2026-09-25 00:43:20 on dist-test-slave-w8nh"
server_key: "aa837d14f147932d6a09671b246553f3"
server_key_iv: "4e8a2c03e14d3ff0163126389f0a9609"
server_key_version: "encryptionkey@0"
I20260925 00:43:20.438277 11656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:20.457847 11656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:20.458606 11656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:20.458773 11656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:20.466132 11656 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:39771
I20260925 00:43:20.466156 11722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:39771 every 8 connection(s)
I20260925 00:43:20.467140 11656 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:20.468820 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 11656
I20260925 00:43:20.468921 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:20.469226 6433 external_mini_cluster.cc:1428] Setting key 80a9573edb6db90740234d310e4f79d9
I20260925 00:43:20.470702 11723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:20.476176 11723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6: Bootstrap starting.
I20260925 00:43:20.478379 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:20.470597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33650 (local address 127.6.72.126:39771)
0925 00:43:20.471257 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:20.471272 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:20.471307 (+ 35us) server_negotiation.cc:407] Connection header received
0925 00:43:20.471951 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:20.471967 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:20.472440 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:20.472775 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:20.473199 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.474157 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.475036 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.475512 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.475684 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:20.476667 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:20.476672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:20.476685 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:20.477075 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:20.477131 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:20.477151 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:20.477280 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:20.477400 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 00:43:20.477661 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":151,"threads_started":1}
I20260925 00:43:20.478533 11723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:20.479267 11723 log.cc:824] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:20.481420 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:43:20.481624 11723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6: No bootstrap required, opened a new log
I20260925 00:43:20.485502 11723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf1788d7334495b1038793424499d6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39771 } }
I20260925 00:43:20.485740 11723 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:20.485814 11723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6faf1788d7334495b1038793424499d6, State: Initialized, Role: FOLLOWER
I20260925 00:43:20.486292 11723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf1788d7334495b1038793424499d6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39771 } }
I20260925 00:43:20.486471 11723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:20.486538 11723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:20.486635 11723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:20.487654 11723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf1788d7334495b1038793424499d6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39771 } }
I20260925 00:43:20.488076 11723 leader_election.cc:302] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6faf1788d7334495b1038793424499d6; no voters:
I20260925 00:43:20.488461 11723 leader_election.cc:288] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:20.488618 11728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:20.488874 11728 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6faf1788d7334495b1038793424499d6, State: Running, Role: LEADER
I20260925 00:43:20.489185 11728 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf1788d7334495b1038793424499d6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39771 } }
I20260925 00:43:20.489715 11723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:20.490931 11729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6faf1788d7334495b1038793424499d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf1788d7334495b1038793424499d6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39771 } } }
I20260925 00:43:20.490936 11730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6faf1788d7334495b1038793424499d6. Latest consensus state: current_term: 1 leader_uuid: "6faf1788d7334495b1038793424499d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6faf1788d7334495b1038793424499d6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39771 } } }
I20260925 00:43:20.491086 11730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:20.491086 11729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:20.491544 11737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:20.494745 11737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:20.500478 11737 catalog_manager.cc:1347] Generated new cluster ID: eab460e19f3045d49c388703270e0772
I20260925 00:43:20.500564 11737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:20.510435 11737 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:20.511336 11737 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:20.517050 11737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6faf1788d7334495b1038793424499d6: Generated new TSK 0
I20260925 00:43:20.517786 11737 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 00:43:20.539995 11753 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 00:43:20.540109 11753 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 00:43:20.540925 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:20.524881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.6.72.126:39771)
0925 00:43:20.524998 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:20.525003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:20.525155 (+ 152us) server_negotiation.cc:407] Connection header received
0925 00:43:20.525319 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:20.525322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:20.525379 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:20.525496 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0925 00:43:20.525712 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.526524 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.527377 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.527726 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.527831 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:20.528104 (+ 273us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 00:43:20.540460 (+ 12356us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40411/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 00:43:20.540762 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33664: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40411/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":28}
W20260925 00:43:20.540995 11726 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33664: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40411/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 00:43:20.541025 11752 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:20.524729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:39771 (local address 127.0.0.1:33664)
0925 00:43:20.525072 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:20.525084 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:20.525180 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:20.525501 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:20.525503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:20.525508 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0925 00:43:20.525613 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:20.525618 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.526651 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.526653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:20.527267 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:20.527274 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.527837 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.527840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:20.527906 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:20.527928 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 00:43:20.540677 (+ 12749us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40411/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 00:43:20.540852 (+ 175us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.72.126:39771: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40411/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":105,"threads_started":1}
I20260925 00:43:20.541460 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 11656
2026-09-25T00:43:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (299 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 00:43:20.582757 6433 mini_oidc.cc:152] Starting JWKS server
I20260925 00:43:20.583763 6433 webserver.cc:460] Webserver started at https://127.0.0.1:35435/ using document root <none> and password file <none>
I20260925 00:43:20.583904 6433 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 00:43:20.584199 6433 webserver.cc:460] Webserver started at http://127.0.0.1:41747/ using document root <none> and password file <none>
2026-09-25T00:43:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:20Z Disabled control of system clock
I20260925 00:43:20.594105 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:42899
--webserver_interface=127.6.72.126
--webserver_port=0
--builtin_ntp_servers=127.6.72.84:36491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:42899
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35435/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 00:43:20.706388 11759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:20.706696 11759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:20.706741 11759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:20.710983 11759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:20.711052 11759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:20.711068 11759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:20.711086 11759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:20.711102 11759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 00:43:20.711118 11759 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 00:43:20.711133 11759 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35435/jwks.json
I20260925 00:43:20.716001 11759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:42899
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:42899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35435/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:20.717221 11759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:20.718673 11759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:20.727818 11764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:20.727838 11767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:20.727842 11765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:20.728019 11759 server_base.cc:1034] running on GCE node
I20260925 00:43:20.728704 11759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:20.730067 11759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:20.731263 11759 hybrid_clock.cc:648] HybridClock initialized: now 1790297000731252 us; error 29 us; skew 500 ppm
I20260925 00:43:20.733276 11759 webserver.cc:460] Webserver started at http://127.6.72.126:36755/ using document root <none> and password file <none>
I20260925 00:43:20.733922 11759 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:20.734031 11759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:20.734297 11759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:20.736622 11759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "94c26b8f881943a38d4c34694f3d4c4f"
format_stamp: "Formatted at 2026-09-25 00:43:20 on dist-test-slave-w8nh"
server_key: "51fd83929896c97896fa369f99b01732"
server_key_iv: "11b35cf99ddef9cb182b7798682e4bf2"
server_key_version: "encryptionkey@0"
I20260925 00:43:20.737253 11759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "94c26b8f881943a38d4c34694f3d4c4f"
format_stamp: "Formatted at 2026-09-25 00:43:20 on dist-test-slave-w8nh"
server_key: "51fd83929896c97896fa369f99b01732"
server_key_iv: "11b35cf99ddef9cb182b7798682e4bf2"
server_key_version: "encryptionkey@0"
I20260925 00:43:20.741340 11759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 00:43:20.743952 11773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:20.745260 11759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:20.745409 11759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "94c26b8f881943a38d4c34694f3d4c4f"
format_stamp: "Formatted at 2026-09-25 00:43:20 on dist-test-slave-w8nh"
server_key: "51fd83929896c97896fa369f99b01732"
server_key_iv: "11b35cf99ddef9cb182b7798682e4bf2"
server_key_version: "encryptionkey@0"
I20260925 00:43:20.745536 11759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:20.759698 11759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:20.760561 11759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:20.760773 11759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:20.767911 11759 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:42899
I20260925 00:43:20.767936 11825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:42899 every 8 connection(s)
I20260925 00:43:20.768942 11759 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:20.770480 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 11759
I20260925 00:43:20.770633 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:20.770928 6433 external_mini_cluster.cc:1428] Setting key 7bd7a9b8b2bce352bcd01cb5b39a3d18
I20260925 00:43:20.772716 11826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:20.778486 11826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f: Bootstrap starting.
I20260925 00:43:20.780328 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:20.772399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54462 (local address 127.6.72.126:42899)
0925 00:43:20.773092 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:20.773109 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:20.773147 (+ 38us) server_negotiation.cc:407] Connection header received
0925 00:43:20.773813 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:20.773830 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:20.774242 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:20.774585 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:20.775013 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.775978 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.776939 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.777409 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.777579 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:20.778509 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:20.778514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:20.778525 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:20.778960 (+ 435us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:20.779014 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:20.779034 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:20.779146 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:20.779299 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 00:43:20.779555 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":138,"threads_started":1}
I20260925 00:43:20.780951 11826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:20.781694 11826 log.cc:824] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:20.783257 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:43:20.783833 11826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f: No bootstrap required, opened a new log
I20260925 00:43:20.787675 11826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c26b8f881943a38d4c34694f3d4c4f" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42899 } }
I20260925 00:43:20.787949 11826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:20.787986 11826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94c26b8f881943a38d4c34694f3d4c4f, State: Initialized, Role: FOLLOWER
I20260925 00:43:20.788424 11826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c26b8f881943a38d4c34694f3d4c4f" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42899 } }
I20260925 00:43:20.788550 11826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:20.788599 11826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:20.788672 11826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:20.789623 11826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c26b8f881943a38d4c34694f3d4c4f" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42899 } }
I20260925 00:43:20.789952 11826 leader_election.cc:302] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94c26b8f881943a38d4c34694f3d4c4f; no voters:
I20260925 00:43:20.790236 11826 leader_election.cc:288] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:20.790429 11831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:20.790656 11831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [term 1 LEADER]: Becoming Leader. State: Replica: 94c26b8f881943a38d4c34694f3d4c4f, State: Running, Role: LEADER
I20260925 00:43:20.791033 11831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c26b8f881943a38d4c34694f3d4c4f" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42899 } }
I20260925 00:43:20.791709 11826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:20.792934 11833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94c26b8f881943a38d4c34694f3d4c4f. Latest consensus state: current_term: 1 leader_uuid: "94c26b8f881943a38d4c34694f3d4c4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c26b8f881943a38d4c34694f3d4c4f" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42899 } } }
I20260925 00:43:20.792949 11832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94c26b8f881943a38d4c34694f3d4c4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c26b8f881943a38d4c34694f3d4c4f" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42899 } } }
I20260925 00:43:20.793179 11832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:20.793164 11833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:20.793797 11840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:20.797144 11840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:20.802817 11840 catalog_manager.cc:1347] Generated new cluster ID: 698f863f3ce2449990b05c246a5f4777
I20260925 00:43:20.802901 11840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:20.816104 11840 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:20.817444 11840 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:20.830562 11840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94c26b8f881943a38d4c34694f3d4c4f: Generated new TSK 0
I20260925 00:43:20.831493 11840 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:43:20.851178 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:20.846003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54478 (local address 127.6.72.126:42899)
0925 00:43:20.846133 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:20.846137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:20.846261 (+ 124us) server_negotiation.cc:407] Connection header received
0925 00:43:20.846411 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:20.846415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:20.846471 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:20.846588 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0925 00:43:20.846814 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.847636 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.848583 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.848931 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.849033 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:20.849287 (+ 254us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 00:43:20.850791 (+ 1504us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 00:43:20.851028 (+ 237us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54478: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":53}
W20260925 00:43:20.851265 11829 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54478: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 00:43:20.852897 11855 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:20.845885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42899 (local address 127.0.0.1:54478)
0925 00:43:20.846164 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:20.846177 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:20.846277 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:20.846597 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:20.846599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:20.846605 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0925 00:43:20.846712 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:20.846717 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.847758 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.847761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:20.848477 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:20.848485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:20.849039 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:20.849042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:20.849101 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:20.849124 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 00:43:20.850956 (+ 1832us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 00:43:20.852719 (+ 1763us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.72.126:42899: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":1576,"thread_start_us":109,"threads_started":1}
I20260925 00:43:20.853626 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 11759
E20260925 00:43:20.861954 11856 webserver.cc:533] Webserver: sslize failed:
E20260925 00:43:20.862032 11856 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-25T00:43:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T00:43:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:20Z Disabled control of system clock
I20260925 00:43:20.900123 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41025
--webserver_interface=127.6.72.126
--webserver_port=0
--builtin_ntp_servers=127.6.72.84:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:41025
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/testchainca.cert with env {}
W20260925 00:43:21.010428 11860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:21.010703 11860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:21.010744 11860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:21.014809 11860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:21.014868 11860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:21.014884 11860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:21.014901 11860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:21.014917 11860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 00:43:21.014933 11860 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 00:43:21.014946 11860 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 00:43:21.020315 11860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:41025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:21.021759 11860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:21.023141 11860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:21.031747 11868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:21.031754 11866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:21.031773 11865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:21.032033 11860 server_base.cc:1034] running on GCE node
I20260925 00:43:21.032701 11860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:21.034102 11860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:21.035261 11860 hybrid_clock.cc:648] HybridClock initialized: now 1790297001035247 us; error 25 us; skew 500 ppm
I20260925 00:43:21.037410 11860 webserver.cc:460] Webserver started at http://127.6.72.126:46789/ using document root <none> and password file <none>
I20260925 00:43:21.038002 11860 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:21.038064 11860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:21.038312 11860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:21.040531 11860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "5b40b4ffc1984d56906e0ed935712427"
format_stamp: "Formatted at 2026-09-25 00:43:21 on dist-test-slave-w8nh"
server_key: "e1c5f574e3da9e596dc9eddd2d7c893a"
server_key_iv: "7de368e42dccaa34d4378c5617020d53"
server_key_version: "encryptionkey@0"
I20260925 00:43:21.041307 11860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "5b40b4ffc1984d56906e0ed935712427"
format_stamp: "Formatted at 2026-09-25 00:43:21 on dist-test-slave-w8nh"
server_key: "e1c5f574e3da9e596dc9eddd2d7c893a"
server_key_iv: "7de368e42dccaa34d4378c5617020d53"
server_key_version: "encryptionkey@0"
I20260925 00:43:21.045280 11860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 00:43:21.047752 11874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:21.049053 11860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 00:43:21.049281 11860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "5b40b4ffc1984d56906e0ed935712427"
format_stamp: "Formatted at 2026-09-25 00:43:21 on dist-test-slave-w8nh"
server_key: "e1c5f574e3da9e596dc9eddd2d7c893a"
server_key_iv: "7de368e42dccaa34d4378c5617020d53"
server_key_version: "encryptionkey@0"
I20260925 00:43:21.049407 11860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:21.064450 11860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:21.065207 11860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:21.065420 11860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:21.072129 11860 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:41025
I20260925 00:43:21.072155 11926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:41025 every 8 connection(s)
I20260925 00:43:21.073107 11860 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:21.075750 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 11860
I20260925 00:43:21.075876 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:21.076175 6433 external_mini_cluster.cc:1428] Setting key cbefdf5ec9f0b47347e3c7f70756a310
I20260925 00:43:21.076491 11927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:21.082897 11927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427: Bootstrap starting.
I20260925 00:43:21.085249 11927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:21.085683 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:21.077640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56866 (local address 127.6.72.126:41025)
0925 00:43:21.078296 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:21.078311 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:21.078347 (+ 36us) server_negotiation.cc:407] Connection header received
0925 00:43:21.079317 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:21.079335 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:21.079716 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:21.080050 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:21.080505 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:21.081479 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:21.082362 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:21.082824 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:21.082995 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:21.083887 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:21.083892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:21.083906 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:21.084349 (+ 443us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:21.084405 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:21.084425 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:21.084534 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:21.084657 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 00:43:21.084922 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":141,"threads_started":1}
I20260925 00:43:21.086041 11927 log.cc:824] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:21.088085 11927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427: No bootstrap required, opened a new log
I20260925 00:43:21.088510 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:43:21.091912 11927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b40b4ffc1984d56906e0ed935712427" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41025 } }
I20260925 00:43:21.092176 11927 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:21.092284 11927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b40b4ffc1984d56906e0ed935712427, State: Initialized, Role: FOLLOWER
I20260925 00:43:21.092772 11927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b40b4ffc1984d56906e0ed935712427" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41025 } }
I20260925 00:43:21.092897 11927 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:21.092986 11927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:21.093091 11927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:21.094028 11927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b40b4ffc1984d56906e0ed935712427" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41025 } }
I20260925 00:43:21.094331 11927 leader_election.cc:302] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b40b4ffc1984d56906e0ed935712427; no voters:
I20260925 00:43:21.094600 11927 leader_election.cc:288] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:21.094730 11932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:21.094974 11932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [term 1 LEADER]: Becoming Leader. State: Replica: 5b40b4ffc1984d56906e0ed935712427, State: Running, Role: LEADER
I20260925 00:43:21.095283 11932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b40b4ffc1984d56906e0ed935712427" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41025 } }
I20260925 00:43:21.095944 11927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:21.097082 11934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b40b4ffc1984d56906e0ed935712427. Latest consensus state: current_term: 1 leader_uuid: "5b40b4ffc1984d56906e0ed935712427" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b40b4ffc1984d56906e0ed935712427" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41025 } } }
I20260925 00:43:21.097244 11934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:21.097661 11941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:21.097592 11933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b40b4ffc1984d56906e0ed935712427" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b40b4ffc1984d56906e0ed935712427" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41025 } } }
I20260925 00:43:21.098160 11933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:21.100487 11941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:21.105971 11941 catalog_manager.cc:1347] Generated new cluster ID: bd6b88130b2c401abef7ac1d75261e38
I20260925 00:43:21.106043 11941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:21.141753 11941 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:21.142699 11941 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:21.150928 11941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b40b4ffc1984d56906e0ed935712427: Generated new TSK 0
I20260925 00:43:21.151669 11941 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:43:21.990109 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:21.161646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56878 (local address 127.6.72.126:41025)
0925 00:43:21.161786 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:21.161791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:21.161965 (+ 174us) server_negotiation.cc:407] Connection header received
0925 00:43:21.162131 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:21.162134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:21.162185 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:21.162286 (+ 101us) server_negotiation.cc:226] Negotiated authn=JWT
0925 00:43:21.162535 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:21.163048 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:21.164354 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:21.164592 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:21.164678 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:21.165107 (+ 429us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 00:43:21.989559 (+824452us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 00:43:21.989937 (+ 378us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56878: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":54}
W20260925 00:43:21.990213 11930 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56878: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 00:43:21.990298 11956 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:21.161558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41025 (local address 127.0.0.1:56878)
0925 00:43:21.161872 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:21.161888 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:21.162004 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:21.162302 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:21.162305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:21.162313 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0925 00:43:21.162429 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:21.162436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:21.163153 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:21.163156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:21.164177 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:21.164187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:21.164715 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:21.164719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:21.164816 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:21.164848 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 00:43:21.989818 (+824970us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 00:43:21.990028 (+ 210us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.72.126:41025: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":107,"threads_started":1}
I20260925 00:43:21.990777 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 11860
2026-09-25T00:43:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1118 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (128 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (126 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:22 dist-test-slave-w8nh krb5kdc[11968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:22 dist-test-slave-w8nh krb5kdc[11968](info): set up 2 sockets
Sep 25 00:43:22 dist-test-slave-w8nh krb5kdc[11968](info): commencing operation
krb5kdc: starting...
W20260925 00:43:23.297499 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:23 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:43:23.423930 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:33109
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:33109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:23.535931 11981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:23.536322 11981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:23.536396 11981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:23.540535 11981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:23.540601 11981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:23.540627 11981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:23.540644 11981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:23.540660 11981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:23.546193 11981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:33109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:33109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:23.547286 11981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:23.548302 11981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 00:43:23.556831 11981 server_base.cc:1034] running on GCE node
W20260925 00:43:23.556799 11989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:23.556802 11986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:23.556806 11987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:23.557636 11981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:23.558806 11981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:23.560027 11981 hybrid_clock.cc:648] HybridClock initialized: now 1790297003559977 us; error 63 us; skew 500 ppm
Sep 25 00:43:23 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297003, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:23.567938 11981 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:43:23.569211 11981 webserver.cc:460] Webserver started at http://127.6.72.126:34435/ using document root <none> and password file <none>
I20260925 00:43:23.569823 11981 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:23.569921 11981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:23.570169 11981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:23.572395 11981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "4451688f7c7e4253a3c516cb801242ca"
format_stamp: "Formatted at 2026-09-25 00:43:23 on dist-test-slave-w8nh"
server_key: "34895010e92ff73ec59ee797993e3276"
server_key_iv: "ccfcb669eab921b2b6dfc91700ec1e94"
server_key_version: "encryptionkey@0"
I20260925 00:43:23.572988 11981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "4451688f7c7e4253a3c516cb801242ca"
format_stamp: "Formatted at 2026-09-25 00:43:23 on dist-test-slave-w8nh"
server_key: "34895010e92ff73ec59ee797993e3276"
server_key_iv: "ccfcb669eab921b2b6dfc91700ec1e94"
server_key_version: "encryptionkey@0"
I20260925 00:43:23.576923 11981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 00:43:23.579831 11996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:23.581357 11981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:23.581506 11981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "4451688f7c7e4253a3c516cb801242ca"
format_stamp: "Formatted at 2026-09-25 00:43:23 on dist-test-slave-w8nh"
server_key: "34895010e92ff73ec59ee797993e3276"
server_key_iv: "ccfcb669eab921b2b6dfc91700ec1e94"
server_key_version: "encryptionkey@0"
I20260925 00:43:23.581633 11981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:23.601589 11981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:23.604765 11981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:23.604991 11981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:23.612346 11981 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:33109
I20260925 00:43:23.612403 12048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:33109 every 8 connection(s)
I20260925 00:43:23.613322 11981 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:23.616150 12049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:23.620785 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 11981
I20260925 00:43:23.620895 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:23.621170 6433 external_mini_cluster.cc:1428] Setting key 1ea37a3ac305dd14efb4cdbdb314185c
I20260925 00:43:23.622232 12049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca: Bootstrap starting.
I20260925 00:43:23.624758 12049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:23.625554 12049 log.cc:824] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:23.627574 12049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca: No bootstrap required, opened a new log
Sep 25 00:43:23 dist-test-slave-w8nh krb5kdc[11968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:23.631868 12049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4451688f7c7e4253a3c516cb801242ca" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33109 } }
I20260925 00:43:23.632117 12049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:23.632201 12049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4451688f7c7e4253a3c516cb801242ca, State: Initialized, Role: FOLLOWER
I20260925 00:43:23.632786 12049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4451688f7c7e4253a3c516cb801242ca" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33109 } }
I20260925 00:43:23.632910 12049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:23.632997 12049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:23.633119 12049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:23.634356 12049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4451688f7c7e4253a3c516cb801242ca" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33109 } }
I20260925 00:43:23.634769 12049 leader_election.cc:302] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4451688f7c7e4253a3c516cb801242ca; no voters:
I20260925 00:43:23.635197 12049 leader_election.cc:288] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:23.635310 12054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:23.635560 12054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [term 1 LEADER]: Becoming Leader. State: Replica: 4451688f7c7e4253a3c516cb801242ca, State: Running, Role: LEADER
I20260925 00:43:23.635957 12054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4451688f7c7e4253a3c516cb801242ca" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33109 } }
I20260925 00:43:23.636667 12049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:23.636777 12052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:23.622660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34460 (local address 127.6.72.126:33109)
0925 00:43:23.623105 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:23.623121 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:23.623155 (+ 34us) server_negotiation.cc:407] Connection header received
0925 00:43:23.623856 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:23.623874 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:23.624302 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:23.624664 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:23.625486 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:23.626531 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:23.627402 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:23.627942 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:23.628114 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:23.631042 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:23.631048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:23.631063 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:23.631093 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:23.633116 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:23.633628 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:23.633633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:23.633639 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:23.633757 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:23.634041 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:23.634043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:23.634045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:23.634562 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:23.634712 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:23.635065 (+ 353us) server_negotiation.cc:299] Negotiation successful
0925 00:43:23.635626 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":140,"threads_started":1}
I20260925 00:43:23.637858 12056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4451688f7c7e4253a3c516cb801242ca. Latest consensus state: current_term: 1 leader_uuid: "4451688f7c7e4253a3c516cb801242ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4451688f7c7e4253a3c516cb801242ca" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33109 } } }
I20260925 00:43:23.637984 12056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:23.638306 12055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4451688f7c7e4253a3c516cb801242ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4451688f7c7e4253a3c516cb801242ca" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33109 } } }
I20260925 00:43:23.638382 12055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:23.639340 12063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:23.642269 12063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:23.648708 12063 catalog_manager.cc:1347] Generated new cluster ID: a73d68f6202240cf969455eb90b76aeb
I20260925 00:43:23.648800 12063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:23.654403 12063 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:23.655328 12063 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:23.667725 12063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4451688f7c7e4253a3c516cb801242ca: Generated new TSK 0
I20260925 00:43:23.668624 12063 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:43:23.699990 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33109
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:23.814738 12075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:23.815033 12075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:23.815079 12075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:23.819315 12075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:23.819391 12075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:23.819494 12075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:23.825213 12075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:23.826354 12075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:23.827423 12075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:23.837080 12083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:23.837080 12080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:23.837080 12081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:23.837436 12075 server_base.cc:1034] running on GCE node
I20260925 00:43:23.838101 12075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:23.838830 12075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:23.839988 12075 hybrid_clock.cc:648] HybridClock initialized: now 1790297003839975 us; error 33 us; skew 500 ppm
Sep 25 00:43:23 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297003, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:23.842878 12075 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:43:23.843726 12075 webserver.cc:460] Webserver started at http://127.6.72.65:42637/ using document root <none> and password file <none>
I20260925 00:43:23.844317 12075 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:23.844429 12075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:23.844602 12075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:23.846657 12075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "569a2337a5144b958edced11dceb726e"
format_stamp: "Formatted at 2026-09-25 00:43:23 on dist-test-slave-w8nh"
server_key: "08f95d4864f509ba873c8a270938a986"
server_key_iv: "f5cb98f4e571142324391e5ac8856d0d"
server_key_version: "encryptionkey@0"
I20260925 00:43:23.847179 12075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "569a2337a5144b958edced11dceb726e"
format_stamp: "Formatted at 2026-09-25 00:43:23 on dist-test-slave-w8nh"
server_key: "08f95d4864f509ba873c8a270938a986"
server_key_iv: "f5cb98f4e571142324391e5ac8856d0d"
server_key_version: "encryptionkey@0"
I20260925 00:43:23.851138 12075 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:43:23.853658 12090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:23.854980 12075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:23.855118 12075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "569a2337a5144b958edced11dceb726e"
format_stamp: "Formatted at 2026-09-25 00:43:23 on dist-test-slave-w8nh"
server_key: "08f95d4864f509ba873c8a270938a986"
server_key_iv: "f5cb98f4e571142324391e5ac8856d0d"
server_key_version: "encryptionkey@0"
I20260925 00:43:23.855238 12075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:23.876215 12075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:23.879383 12075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:23.879604 12075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:23.880216 12075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:23.881217 12075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:23.881278 12075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:23.881330 12075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:23.881352 12075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:23.891109 12075 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:39645
I20260925 00:43:23.891126 12203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:39645 every 8 connection(s)
I20260925 00:43:23.892053 12075 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:23.896816 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 12075
I20260925 00:43:23.896947 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:23.897298 6433 external_mini_cluster.cc:1428] Setting key 22d377624edf2390ad16a00d231283ac
Sep 25 00:43:23 dist-test-slave-w8nh krb5kdc[11968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297003, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:43:23.908347 12052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:23.893926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:38465 (local address 127.6.72.126:33109)
0925 00:43:23.894099 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:23.894104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:23.894921 (+ 817us) server_negotiation.cc:407] Connection header received
0925 00:43:23.896387 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:23.896390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:23.896439 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:23.896540 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:23.898197 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:23.898963 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:23.900852 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:23.901343 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:23.901481 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:23.904449 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:23.904452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:23.904454 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:23.904480 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:23.906029 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:23.906602 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:23.906607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:23.906614 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:23.906698 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:23.907061 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:23.907064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:23.907066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:23.907261 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:23.907368 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:23.907980 (+ 612us) server_negotiation.cc:299] Negotiation successful
0925 00:43:23.908124 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 00:43:23.909282 12206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:23.894334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:33109 (local address 127.6.72.65:38465)
0925 00:43:23.894760 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:23.894803 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:23.896170 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:23.896798 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:23.896808 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:23.897360 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:23.898032 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:23.898053 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:23.899096 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:23.899100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:23.900660 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:23.900679 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:23.901526 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:23.901529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:23.901699 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:23.902389 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:23.902415 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:23.904151 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:23.906176 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:23.906182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:23.906203 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:23.906483 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:23.906794 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:23.906797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:23.906799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:23.906952 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:23.907371 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:23.907381 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:23.907718 (+ 337us) client_negotiation.cc:769] Sending connection context
0925 00:43:23.907917 (+ 199us) client_negotiation.cc:240] Negotiation successful
0925 00:43:23.908339 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:43:23.910888 12204 heartbeater.cc:347] Connected to a master server at 127.6.72.126:33109
I20260925 00:43:23.911163 12204 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:23.911682 12204 heartbeater.cc:511] Master 127.6.72.126:33109 requested a full tablet report, sending...
I20260925 00:43:23.913319 12013 ts_manager.cc:194] Registered new tserver with Master: 569a2337a5144b958edced11dceb726e (127.6.72.65:39645)
I20260925 00:43:23.914485 12013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:38465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:43:23.927688 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33109
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:24.047710 12209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:24.048028 12209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:24.048075 12209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:24.052740 12209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:24.052834 12209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:24.052901 12209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:24.059412 12209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:24.060712 12209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:24.061805 12209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:24.071972 12214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:24.071983 12217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:24.071988 12215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:24.072366 12209 server_base.cc:1034] running on GCE node
I20260925 00:43:24.072842 12209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:24.073570 12209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:24.074764 12209 hybrid_clock.cc:648] HybridClock initialized: now 1790297004074724 us; error 51 us; skew 500 ppm
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:24.077903 12209 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:43:24.079022 12209 webserver.cc:460] Webserver started at http://127.6.72.66:40643/ using document root <none> and password file <none>
I20260925 00:43:24.079656 12209 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:24.079722 12209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:24.079908 12209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:24.082115 12209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "0354fc425a1d4b0589c74d2b83d94d46"
format_stamp: "Formatted at 2026-09-25 00:43:24 on dist-test-slave-w8nh"
server_key: "a698f25f7251f3883628432bcd32ffef"
server_key_iv: "30a9ae87fbfcca70bcaa80e51850c502"
server_key_version: "encryptionkey@0"
I20260925 00:43:24.082669 12209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "0354fc425a1d4b0589c74d2b83d94d46"
format_stamp: "Formatted at 2026-09-25 00:43:24 on dist-test-slave-w8nh"
server_key: "a698f25f7251f3883628432bcd32ffef"
server_key_iv: "30a9ae87fbfcca70bcaa80e51850c502"
server_key_version: "encryptionkey@0"
I20260925 00:43:24.086781 12209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:43:24.089445 12224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:24.090937 12209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:24.091058 12209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "0354fc425a1d4b0589c74d2b83d94d46"
format_stamp: "Formatted at 2026-09-25 00:43:24 on dist-test-slave-w8nh"
server_key: "a698f25f7251f3883628432bcd32ffef"
server_key_iv: "30a9ae87fbfcca70bcaa80e51850c502"
server_key_version: "encryptionkey@0"
I20260925 00:43:24.091167 12209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:24.103480 12209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:24.106632 12209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:24.106801 12209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:24.107420 12209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:24.108234 12209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:24.108359 12209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:24.108403 12209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:24.108423 12209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:24.117501 12209 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:46767
I20260925 00:43:24.117524 12337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:46767 every 8 connection(s)
I20260925 00:43:24.118453 12209 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:24.124980 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 12209
I20260925 00:43:24.125125 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:24.125427 6433 external_mini_cluster.cc:1428] Setting key 8cb2d875587bd9a21c026901e718d5c5
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:24.134390 12052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:24.120500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:52425 (local address 127.6.72.126:33109)
0925 00:43:24.120704 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:24.120709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:24.121930 (+ 1221us) server_negotiation.cc:407] Connection header received
0925 00:43:24.123286 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:24.123289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:24.123337 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:24.123413 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:24.124646 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.125286 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.126820 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.127150 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.127269 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:24.130304 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:24.130311 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:24.130318 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:24.130351 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:24.132075 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:24.132669 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:24.132674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:24.132678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:24.132778 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:24.133122 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:24.133128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:24.133132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:24.133408 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:24.133525 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:24.134035 (+ 510us) server_negotiation.cc:299] Negotiation successful
0925 00:43:24.134180 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 00:43:24.135143 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:24.121187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:33109 (local address 127.6.72.66:52425)
0925 00:43:24.121764 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:24.121822 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:24.123079 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:24.123548 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:24.123555 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:24.123928 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:24.124487 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:24.124502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.125415 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.125419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:24.126605 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:24.126615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.127307 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.127312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:24.127506 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:24.128210 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:24.128227 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:24.130126 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:24.132200 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:24.132205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:24.132219 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:24.132534 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:24.132885 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:24.132887 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:24.132888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:24.133016 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:24.133519 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:24.133524 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:24.133766 (+ 242us) client_negotiation.cc:769] Sending connection context
0925 00:43:24.133917 (+ 151us) client_negotiation.cc:240] Negotiation successful
0925 00:43:24.134179 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":142,"threads_started":1}
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:43:24.136919 12338 heartbeater.cc:347] Connected to a master server at 127.6.72.126:33109
I20260925 00:43:24.137234 12338 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:24.137804 12338 heartbeater.cc:511] Master 127.6.72.126:33109 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:43:24.139140 12013 ts_manager.cc:194] Registered new tserver with Master: 0354fc425a1d4b0589c74d2b83d94d46 (127.6.72.66:46767)
I20260925 00:43:24.139726 12013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:52425
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:43:24.155884 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33109
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:24.269677 12343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:24.269953 12343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:24.270000 12343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:24.274238 12343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:24.274318 12343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:24.274384 12343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:24.279949 12343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:24.281188 12343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:24.282199 12343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:24.292197 12351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:24.292259 12348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:24.292258 12349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:24.292421 12343 server_base.cc:1034] running on GCE node
I20260925 00:43:24.292920 12343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:24.293685 12343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:24.294912 12343 hybrid_clock.cc:648] HybridClock initialized: now 1790297004294871 us; error 53 us; skew 500 ppm
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:24.298242 12343 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:43:24.299392 12343 webserver.cc:460] Webserver started at http://127.6.72.67:35257/ using document root <none> and password file <none>
I20260925 00:43:24.300120 12343 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:24.300230 12343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:24.300489 12343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:24.302767 12343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "4a2d8d14e2df420e904367785790ee08"
format_stamp: "Formatted at 2026-09-25 00:43:24 on dist-test-slave-w8nh"
server_key: "270d2dc5c5405e92f8913f687f0f1c5d"
server_key_iv: "fbfd2867fc601e221269d389321a1678"
server_key_version: "encryptionkey@0"
I20260925 00:43:24.303393 12343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "4a2d8d14e2df420e904367785790ee08"
format_stamp: "Formatted at 2026-09-25 00:43:24 on dist-test-slave-w8nh"
server_key: "270d2dc5c5405e92f8913f687f0f1c5d"
server_key_iv: "fbfd2867fc601e221269d389321a1678"
server_key_version: "encryptionkey@0"
I20260925 00:43:24.307569 12343 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260925 00:43:24.310426 12358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:24.311882 12343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:24.312069 12343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "4a2d8d14e2df420e904367785790ee08"
format_stamp: "Formatted at 2026-09-25 00:43:24 on dist-test-slave-w8nh"
server_key: "270d2dc5c5405e92f8913f687f0f1c5d"
server_key_iv: "fbfd2867fc601e221269d389321a1678"
server_key_version: "encryptionkey@0"
I20260925 00:43:24.312244 12343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:24.327215 12343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:24.330744 12343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:24.330992 12343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:24.331647 12343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:24.332638 12343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:24.332713 12343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:24.332787 12343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:24.332841 12343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:24.342101 12343 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:43353
I20260925 00:43:24.342119 12471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:43353 every 8 connection(s)
I20260925 00:43:24.343030 12343 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:24.353266 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 12343
I20260925 00:43:24.353404 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:24.353830 6433 external_mini_cluster.cc:1428] Setting key 0d2707efef6a74b8d2bb154255253677
I20260925 00:43:24.357618 12052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:24.344914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:37997 (local address 127.6.72.126:33109)
0925 00:43:24.345073 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:24.345077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:24.345964 (+ 887us) server_negotiation.cc:407] Connection header received
0925 00:43:24.347290 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:24.347293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:24.347340 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:24.347417 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:24.348668 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.349207 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.350274 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.350476 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.350549 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:24.353136 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:24.353140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:24.353144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:24.353172 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:24.355130 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:24.355868 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:24.355870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:24.355872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:24.355954 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:24.356440 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:24.356444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:24.356445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:24.356721 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:24.356784 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:24.357365 (+ 581us) server_negotiation.cc:299] Negotiation successful
0925 00:43:24.357468 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 00:43:24.358566 12474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:24.345351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:33109 (local address 127.6.72.67:37997)
0925 00:43:24.345804 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:24.345848 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:24.347081 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:24.347580 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:24.347586 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:24.347965 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:24.348534 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:24.348548 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.349310 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.349313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:24.350174 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:24.350182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.350599 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.350601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:24.350745 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:24.351401 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:24.351419 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:24.352962 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:24.355364 (+ 2402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:24.355370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:24.355387 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:24.355741 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:24.356052 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:24.356055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:24.356057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:24.356284 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:24.356851 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:24.356857 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:24.357101 (+ 244us) client_negotiation.cc:769] Sending connection context
0925 00:43:24.357286 (+ 185us) client_negotiation.cc:240] Negotiation successful
0925 00:43:24.357569 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":113,"threads_started":1}
I20260925 00:43:24.359828 12472 heartbeater.cc:347] Connected to a master server at 127.6.72.126:33109
I20260925 00:43:24.360073 12472 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:24.360647 12472 heartbeater.cc:511] Master 127.6.72.126:33109 requested a full tablet report, sending...
I20260925 00:43:24.361706 12013 ts_manager.cc:194] Registered new tserver with Master: 4a2d8d14e2df420e904367785790ee08 (127.6.72.67:43353)
I20260925 00:43:24.362304 12013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:37997
I20260925 00:43:24.368505 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/test-admin.keytab.
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:24.386438 6433 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 00:43:24.389590 12479 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:24.387843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56776 (local address 127.6.72.65:39645)
0925 00:43:24.388193 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:24.388212 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:24.388267 (+ 55us) server_negotiation.cc:407] Connection header received
0925 00:43:24.388396 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:24.388401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:24.388602 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:24.388773 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:24.389361 (+ 588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56776: BlockingRecv error: recv got EOF from 127.0.0.1:56776 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":65,"threads_started":1}
W20260925 00:43:24.389724 6433 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5cc' of type FILE
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:24.390969 6433 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:24.391984 6433 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:43:24.402981 12479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:24.392968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.6.72.65:39645)
0925 00:43:24.393099 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:24.393104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:24.393321 (+ 217us) server_negotiation.cc:407] Connection header received
0925 00:43:24.393494 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:24.393497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:24.393541 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:24.393628 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:24.394187 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.395120 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.396125 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.396585 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.396709 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:24.399305 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:24.399316 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:24.399331 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:24.399357 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:24.401313 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:24.401765 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:24.401771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:24.401772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:24.401850 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:24.402123 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:24.402125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:24.402126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:24.402397 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:24.402552 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:24.402721 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 00:43:24.402849 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260925 00:43:24.407107 12479 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:24.406057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56786 (local address 127.6.72.65:39645)
0925 00:43:24.406223 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:24.406227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:24.406239 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:43:24.406356 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:24.406358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:24.406394 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:24.406457 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:24.406945 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56786: BlockingRecv error: recv got EOF from 127.0.0.1:56786 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260925 00:43:24.407294 6433 init.cc:244] Error while opening credential cache '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5cc)
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:24.408453 6433 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:24.409323 6433 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:43:24.418251 12479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:24.410287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56802 (local address 127.6.72.65:39645)
0925 00:43:24.410437 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:24.410440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:24.410456 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:24.410577 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:24.410579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:24.410614 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:24.410678 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:24.411165 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.411676 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.412482 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.412730 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.412813 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:24.415017 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:24.415020 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:24.415023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:24.415046 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:24.416663 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:24.417093 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:24.417096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:24.417097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:24.417181 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:24.417426 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:24.417428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:24.417429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:24.417604 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:24.417706 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:24.417986 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 00:43:24.418126 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260925 00:43:24.421195 12479 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:24.420206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56812 (local address 127.6.72.65:39645)
0925 00:43:24.420376 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:24.420380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:24.420490 (+ 110us) server_negotiation.cc:407] Connection header received
0925 00:43:24.420574 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:24.420576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:24.420610 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:24.420672 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:24.421033 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56812: BlockingRecv error: recv got EOF from 127.0.0.1:56812 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260925 00:43:24.421329 6433 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790296971355272-6433-0/krb5kdc/krb5cc' of type FILE
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:24.422386 6433 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:24.423277 6433 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[11968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:43:24.432119 12479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:24.424300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.6.72.65:39645)
0925 00:43:24.424433 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:24.424438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:24.424548 (+ 110us) server_negotiation.cc:407] Connection header received
0925 00:43:24.424704 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:24.424708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:24.424756 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:24.424838 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:24.425349 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.425852 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.426627 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:24.426844 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:24.426951 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:24.429216 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:24.429219 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:24.429222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:24.429259 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:24.430489 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:24.430917 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:24.430920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:24.430922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:24.431004 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:24.431279 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:24.431283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:24.431285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:24.431457 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:24.431595 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:24.431836 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 00:43:24.431943 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:43:24.433208 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 12075
I20260925 00:43:24.438679 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 12209
I20260925 00:43:24.444012 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 12343
I20260925 00:43:24.449247 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 11981
2026-09-25T00:43:24Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2205 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[12492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[12492](info): set up 2 sockets
Sep 25 00:43:24 dist-test-slave-w8nh krb5kdc[12492](info): commencing operation
krb5kdc: starting...
W20260925 00:43:25.496908 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:25 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.126.keytab.
Entry for principal oryx/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.126.keytab.
I20260925 00:43:25.624699 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:35623
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.126.keytab
--principal=oryx/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:35623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:25.737632 12505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:25.737950 12505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:25.737993 12505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:25.742236 12505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:25.742311 12505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:25.742331 12505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:25.742347 12505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:25.742362 12505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:25.748045 12505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:35623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.126.keytab
--principal=oryx/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:35623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:25.749388 12505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:25.750509 12505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:25.759502 12510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:25.759593 12511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:25.759729 12505 server_base.cc:1034] running on GCE node
W20260925 00:43:25.759570 12513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:25.760491 12505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:25.761818 12505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:25.762992 12505 hybrid_clock.cc:648] HybridClock initialized: now 1790297005762960 us; error 47 us; skew 500 ppm
Sep 25 00:43:25 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297005, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:25.765934 12505 init.cc:377] Logged in from keytab as oryx/127.6.72.126@KRBTEST.COM (short username oryx)
I20260925 00:43:25.766970 12505 webserver.cc:460] Webserver started at http://127.6.72.126:38093/ using document root <none> and password file <none>
I20260925 00:43:25.767580 12505 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:25.767678 12505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:25.767918 12505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:25.770192 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "4c635738475c448680441d391652e4c5"
format_stamp: "Formatted at 2026-09-25 00:43:25 on dist-test-slave-w8nh"
server_key: "ccdc02191cb9b00dc1be91fcfb0772b5"
server_key_iv: "7504df84e6e52738bcb3f56549d52d28"
server_key_version: "encryptionkey@0"
I20260925 00:43:25.770789 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "4c635738475c448680441d391652e4c5"
format_stamp: "Formatted at 2026-09-25 00:43:25 on dist-test-slave-w8nh"
server_key: "ccdc02191cb9b00dc1be91fcfb0772b5"
server_key_iv: "7504df84e6e52738bcb3f56549d52d28"
server_key_version: "encryptionkey@0"
I20260925 00:43:25.774940 12505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 00:43:25.777635 12520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:25.778968 12505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:25.779129 12505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "4c635738475c448680441d391652e4c5"
format_stamp: "Formatted at 2026-09-25 00:43:25 on dist-test-slave-w8nh"
server_key: "ccdc02191cb9b00dc1be91fcfb0772b5"
server_key_iv: "7504df84e6e52738bcb3f56549d52d28"
server_key_version: "encryptionkey@0"
I20260925 00:43:25.779256 12505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:25.802731 12505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:25.806258 12505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:25.806479 12505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:25.814072 12505 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:35623
I20260925 00:43:25.814424 12572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:35623 every 8 connection(s)
I20260925 00:43:25.815126 12505 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:25.818020 12573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:25.821696 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 12505
I20260925 00:43:25.821802 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:25.822103 6433 external_mini_cluster.cc:1428] Setting key e6f6283336939a27eb94bbd6d12d589f
I20260925 00:43:25.824491 12573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5: Bootstrap starting.
I20260925 00:43:25.827407 12573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:25.828346 12573 log.cc:824] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:25.830775 12573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5: No bootstrap required, opened a new log
Sep 25 00:43:25 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:25.834820 12573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } }
I20260925 00:43:25.835112 12573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:25.835167 12573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c635738475c448680441d391652e4c5, State: Initialized, Role: FOLLOWER
I20260925 00:43:25.835685 12573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } }
I20260925 00:43:25.835843 12573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:25.835916 12573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:25.836024 12573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:25.837321 12573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } }
I20260925 00:43:25.837718 12573 leader_election.cc:302] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c635738475c448680441d391652e4c5; no voters:
I20260925 00:43:25.838073 12573 leader_election.cc:288] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:25.838328 12578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:25.838603 12578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 1 LEADER]: Becoming Leader. State: Replica: 4c635738475c448680441d391652e4c5, State: Running, Role: LEADER
I20260925 00:43:25.838899 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:25.823578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39290 (local address 127.6.72.126:35623)
0925 00:43:25.824060 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:25.824079 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:25.824124 (+ 45us) server_negotiation.cc:407] Connection header received
0925 00:43:25.825015 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:25.825039 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:25.825489 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:25.825898 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:25.826602 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:25.828110 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:25.829311 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:25.829983 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:25.830159 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:25.832781 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:25.832788 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:25.832807 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:25.832841 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:25.835414 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:25.835895 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:25.835902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:25.835908 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:25.836055 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:25.836417 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:25.836421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:25.836422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:25.836934 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:25.837148 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:25.837389 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 00:43:25.837852 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":117,"threads_started":1}
I20260925 00:43:25.838936 12578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } }
I20260925 00:43:25.839489 12573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:25.840910 12579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c635738475c448680441d391652e4c5. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } }
I20260925 00:43:25.841063 12579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:25.840907 12580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } }
I20260925 00:43:25.841182 12580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:25.841627 12592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:25.845082 12592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:25.850900 12592 catalog_manager.cc:1347] Generated new cluster ID: 497d8ee0601248caa0af71534968b68a
I20260925 00:43:25.850988 12592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:25.874086 12592 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:25.875114 12592 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:25.891028 12592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5: Generated new TSK 0
I20260925 00:43:25.891906 12592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab.
Entry for principal oryx/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab.
I20260925 00:43:25.923804 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35623
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab
--principal=oryx/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:26.039315 12599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:26.039721 12599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:26.039799 12599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:26.045522 12599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:26.045596 12599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:26.045696 12599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:26.051183 12599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab
--principal=oryx/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:26.052402 12599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:26.053483 12599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:26.063957 12604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:26.063930 12605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:26.063930 12607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:26.063971 12599 server_base.cc:1034] running on GCE node
I20260925 00:43:26.064643 12599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:26.065317 12599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:26.066538 12599 hybrid_clock.cc:648] HybridClock initialized: now 1790297006066504 us; error 46 us; skew 500 ppm
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297006, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:26.069528 12599 init.cc:377] Logged in from keytab as oryx/127.6.72.65@KRBTEST.COM (short username oryx)
I20260925 00:43:26.070451 12599 webserver.cc:460] Webserver started at http://127.6.72.65:35993/ using document root <none> and password file <none>
I20260925 00:43:26.071026 12599 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:26.071090 12599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:26.071257 12599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:26.073323 12599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "51d08a7b3a834c94982a8cb00ee067e9"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "edcfb5207ce0f9d7509e7bf6aedd5fd5"
server_key_iv: "c373090106071015fb9bc4c4efc96e20"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.073837 12599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "51d08a7b3a834c94982a8cb00ee067e9"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "edcfb5207ce0f9d7509e7bf6aedd5fd5"
server_key_iv: "c373090106071015fb9bc4c4efc96e20"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.077523 12599 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:43:26.079895 12614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.081333 12599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:26.081476 12599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "51d08a7b3a834c94982a8cb00ee067e9"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "edcfb5207ce0f9d7509e7bf6aedd5fd5"
server_key_iv: "c373090106071015fb9bc4c4efc96e20"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.081589 12599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:26.094897 12599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:26.098351 12599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:26.098562 12599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:26.099188 12599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:26.100031 12599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:26.100077 12599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.100144 12599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:26.100176 12599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.109705 12599 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:45781
I20260925 00:43:26.109709 12727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:45781 every 8 connection(s)
I20260925 00:43:26.110713 12599 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:26.111050 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 12599
I20260925 00:43:26.111189 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:26.111506 6433 external_mini_cluster.cc:1428] Setting key c7e59f0a56cad3fd7ab451dc84f775ff
WARNING: no policy specified for oryx/127.6.72.66@KRBTEST.COM; defaulting to no policy
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297006, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.66@KRBTEST.COM" created.
I20260925 00:43:26.125387 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.112748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:42247 (local address 127.6.72.126:35623)
0925 00:43:26.112877 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:26.112900 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:26.113938 (+ 1038us) server_negotiation.cc:407] Connection header received
0925 00:43:26.115609 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:26.115621 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:26.115679 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:26.115784 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:26.117072 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.117620 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.118761 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.118985 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.119095 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.121729 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:26.121732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:26.121734 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:26.121758 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:26.123298 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.123806 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.123811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.123813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.123898 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.124283 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.124287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.124288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.124469 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:26.124575 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:26.125084 (+ 509us) server_negotiation.cc:299] Negotiation successful
0925 00:43:26.125214 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 00:43:26.126299 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.113245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.65:42247)
0925 00:43:26.113756 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:26.113815 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:26.115391 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:26.115973 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:26.115981 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:26.116432 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:26.116926 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.116940 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.117751 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.117760 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.118625 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.118635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.119146 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.119151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.119314 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.119965 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:26.119981 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:26.121550 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:26.123414 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.123420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.123438 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.123700 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.123997 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.124000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.124002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.124151 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.124575 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:26.124583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:26.124840 (+ 257us) client_negotiation.cc:769] Sending connection context
0925 00:43:26.125032 (+ 192us) client_negotiation.cc:240] Negotiation successful
0925 00:43:26.125384 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":105,"threads_started":1}
I20260925 00:43:26.127562 12728 heartbeater.cc:347] Connected to a master server at 127.6.72.126:35623
I20260925 00:43:26.127840 12728 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:26.128491 12728 heartbeater.cc:511] Master 127.6.72.126:35623 requested a full tablet report, sending...
I20260925 00:43:26.130195 12537 ts_manager.cc:194] Registered new tserver with Master: 51d08a7b3a834c94982a8cb00ee067e9 (127.6.72.65:45781)
I20260925 00:43:26.131390 12537 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:42247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab.
Entry for principal oryx/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab.
I20260925 00:43:26.140416 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35623
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab
--principal=oryx/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:26.251951 12733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:26.252338 12733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:26.252436 12733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:26.256547 12733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:26.256659 12733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:26.256762 12733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:26.262360 12733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab
--principal=oryx/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:26.263506 12733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:26.264707 12733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:26.274541 12739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:26.274569 12741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:26.274541 12738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:26.274781 12733 server_base.cc:1034] running on GCE node
I20260925 00:43:26.275408 12733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:26.276165 12733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:26.277359 12733 hybrid_clock.cc:648] HybridClock initialized: now 1790297006277342 us; error 29 us; skew 500 ppm
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297006, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:26.280414 12733 init.cc:377] Logged in from keytab as oryx/127.6.72.66@KRBTEST.COM (short username oryx)
I20260925 00:43:26.281399 12733 webserver.cc:460] Webserver started at http://127.6.72.66:33817/ using document root <none> and password file <none>
I20260925 00:43:26.282012 12733 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:26.282078 12733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:26.282348 12733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:26.284633 12733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "b569cd1d3ae043839c852ca947f15c50"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "10db122b29b0893d035a959d20c41b3f"
server_key_iv: "e680c39506af6d718e003a660563ac76"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.285305 12733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "b569cd1d3ae043839c852ca947f15c50"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "10db122b29b0893d035a959d20c41b3f"
server_key_iv: "e680c39506af6d718e003a660563ac76"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.289544 12733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260925 00:43:26.292158 12748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.293569 12733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:26.293711 12733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "b569cd1d3ae043839c852ca947f15c50"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "10db122b29b0893d035a959d20c41b3f"
server_key_iv: "e680c39506af6d718e003a660563ac76"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.293834 12733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:26.324806 12733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:26.328356 12733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:26.328591 12733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:26.329252 12733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:26.330158 12733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:26.330235 12733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.330312 12733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:26.330358 12733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.340782 12733 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:33187
I20260925 00:43:26.340835 12861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:33187 every 8 connection(s)
I20260925 00:43:26.341971 12733 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:26.347405 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 12733
I20260925 00:43:26.347564 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:26.347827 6433 external_mini_cluster.cc:1428] Setting key 3af13801039aa3172970bfb70aee3115
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297006, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:43:26.358417 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.343913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:55643 (local address 127.6.72.126:35623)
0925 00:43:26.344072 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:26.344076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:26.345106 (+ 1030us) server_negotiation.cc:407] Connection header received
0925 00:43:26.346505 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:26.346509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:26.346573 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:26.346685 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:26.348076 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.348822 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.350623 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.350887 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.351060 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.353747 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:26.353752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:26.353755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:26.353786 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:26.355742 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.356548 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.356552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.356553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.356650 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.357072 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.357076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.357080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.357306 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:26.357422 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:26.358070 (+ 648us) server_negotiation.cc:299] Negotiation successful
0925 00:43:26.358220 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 00:43:26.359650 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.344407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.66:55643)
0925 00:43:26.344912 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:26.344968 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:26.346256 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:26.346874 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:26.346882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:26.347288 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:26.347878 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.347916 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.348962 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.348965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.350468 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.350483 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.351121 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.351129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.351320 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.351961 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:26.351978 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:26.353544 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:26.355895 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.355901 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.355917 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.356383 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.356768 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.356771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.356773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.356933 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.357435 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:26.357444 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:26.357788 (+ 344us) client_negotiation.cc:769] Sending connection context
0925 00:43:26.358009 (+ 221us) client_negotiation.cc:240] Negotiation successful
0925 00:43:26.358391 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":115,"threads_started":1}
I20260925 00:43:26.360975 12862 heartbeater.cc:347] Connected to a master server at 127.6.72.126:35623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.67@KRBTEST.COM" created.
I20260925 00:43:26.361287 12862 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:26.361922 12862 heartbeater.cc:511] Master 127.6.72.126:35623 requested a full tablet report, sending...
I20260925 00:43:26.363116 12537 ts_manager.cc:194] Registered new tserver with Master: b569cd1d3ae043839c852ca947f15c50 (127.6.72.66:33187)
I20260925 00:43:26.363759 12537 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:55643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab.
Entry for principal oryx/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab.
I20260925 00:43:26.379338 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35623
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab
--principal=oryx/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:26.494884 12867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:26.495177 12867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:26.495223 12867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:26.499397 12867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:26.499480 12867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:26.499552 12867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:26.505190 12867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab
--principal=oryx/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:26.506270 12867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:26.507277 12867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:26.517205 12875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:26.517143 12872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:26.517167 12873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:26.517421 12867 server_base.cc:1034] running on GCE node
I20260925 00:43:26.517920 12867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:26.518607 12867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:26.519800 12867 hybrid_clock.cc:648] HybridClock initialized: now 1790297006519784 us; error 38 us; skew 500 ppm
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297006, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:26.522687 12867 init.cc:377] Logged in from keytab as oryx/127.6.72.67@KRBTEST.COM (short username oryx)
I20260925 00:43:26.523538 12867 webserver.cc:460] Webserver started at http://127.6.72.67:41179/ using document root <none> and password file <none>
I20260925 00:43:26.524067 12867 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:26.524127 12867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:26.524372 12867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:26.526443 12867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "fd2b33f21d8f40348ee6037d9642dbfd"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "2d85a954273feee1703661d0805b802e"
server_key_iv: "385afa931cf36163795419f882b3fb75"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.526952 12867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "fd2b33f21d8f40348ee6037d9642dbfd"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "2d85a954273feee1703661d0805b802e"
server_key_iv: "385afa931cf36163795419f882b3fb75"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.530845 12867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260925 00:43:26.533380 12882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.534651 12867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:26.534752 12867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "fd2b33f21d8f40348ee6037d9642dbfd"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "2d85a954273feee1703661d0805b802e"
server_key_iv: "385afa931cf36163795419f882b3fb75"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.534845 12867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:26.545018 12867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:26.548359 12867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:26.548514 12867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:26.549064 12867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:26.549835 12867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:26.549883 12867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.549922 12867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:26.549938 12867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.559262 12867 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:46823
I20260925 00:43:26.559275 12995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:46823 every 8 connection(s)
I20260925 00:43:26.560180 12867 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:26.566213 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 12867
I20260925 00:43:26.566381 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:26.566695 6433 external_mini_cluster.cc:1428] Setting key 07af837e0d15c4cb5a1c4bfaaa71aa04
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297006, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:26.574116 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.562169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:47693 (local address 127.6.72.126:35623)
0925 00:43:26.562319 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:26.562323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:26.563155 (+ 832us) server_negotiation.cc:407] Connection header received
0925 00:43:26.564483 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:26.564487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:26.564550 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:26.564631 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:26.565823 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.566394 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.567579 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.567816 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.567933 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.570444 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:26.570448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:26.570450 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:26.570479 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:26.571928 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.572537 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.572541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.572543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.572628 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.572979 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.572982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.572984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.573172 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:26.573280 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:26.573815 (+ 535us) server_negotiation.cc:299] Negotiation successful
0925 00:43:26.573940 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:43:26.574949 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.562534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.67:47693)
0925 00:43:26.562998 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:26.563040 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:26.564227 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:26.564831 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:26.564838 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:26.565193 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:26.565671 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.565684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.566528 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.566531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.567449 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.567459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.567966 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.567970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.568144 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.568827 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:26.568843 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:26.570282 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:26.572062 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.572067 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.572083 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.572434 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.572722 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.572727 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.572729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.572875 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.573298 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:26.573305 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:26.573561 (+ 256us) client_negotiation.cc:769] Sending connection context
0925 00:43:26.573749 (+ 188us) client_negotiation.cc:240] Negotiation successful
0925 00:43:26.574086 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":117,"threads_started":1}
I20260925 00:43:26.576179 12996 heartbeater.cc:347] Connected to a master server at 127.6.72.126:35623
I20260925 00:43:26.576486 12996 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:26.577025 12996 heartbeater.cc:511] Master 127.6.72.126:35623 requested a full tablet report, sending...
I20260925 00:43:26.578130 12537 ts_manager.cc:194] Registered new tserver with Master: fd2b33f21d8f40348ee6037d9642dbfd (127.6.72.67:46823)
I20260925 00:43:26.578773 12537 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:47693
I20260925 00:43:26.581048 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.72.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.125.keytab.
Entry for principal oryx/127.6.72.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.125.keytab.
I20260925 00:43:26.611737 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.125:38209
--webserver_interface=127.6.72.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.125.keytab
--principal=oryx/127.6.72.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:35623,127.6.72.125:38209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:26.727594 13001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:26.727888 13001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:26.727972 13001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:26.732101 13001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:26.732182 13001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:26.732201 13001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:26.732218 13001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:26.732234 13001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:26.737903 13001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:35623,127.6.72.125:38209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.125.keytab
--principal=oryx/127.6.72.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.125:38209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.72.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
W20260925 00:43:26.738225 13001 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.72.126:35623,127.6.72.125:38209'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 00:43:26.739262 13001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:26.740427 13001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:26.749370 13009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:26.749388 13007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:26.749394 13006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:26.749792 13001 server_base.cc:1034] running on GCE node
I20260925 00:43:26.750420 13001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:26.751677 13001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:26.752873 13001 hybrid_clock.cc:648] HybridClock initialized: now 1790297006752837 us; error 52 us; skew 500 ppm
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297006, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:26.755846 13001 init.cc:377] Logged in from keytab as oryx/127.6.72.125@KRBTEST.COM (short username oryx)
I20260925 00:43:26.756994 13001 webserver.cc:460] Webserver started at http://127.6.72.125:45799/ using document root <none> and password file <none>
I20260925 00:43:26.757587 13001 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:26.757652 13001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:26.757824 13001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:26.760026 13001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data/instance:
uuid: "a1af392529814d1cb61dca0eca097f5e"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "21aa4fa2e7e9ca172e30b32eadeec689"
server_key_iv: "c2bf71ce29d52de71547cee18a721354"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.760643 13001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal/instance:
uuid: "a1af392529814d1cb61dca0eca097f5e"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "21aa4fa2e7e9ca172e30b32eadeec689"
server_key_iv: "c2bf71ce29d52de71547cee18a721354"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.764621 13001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:43:26.767323 13016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.768906 13001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:26.769025 13001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal
uuid: "a1af392529814d1cb61dca0eca097f5e"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "21aa4fa2e7e9ca172e30b32eadeec689"
server_key_iv: "c2bf71ce29d52de71547cee18a721354"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.769137 13001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:26.788077 13001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:26.791404 13001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:26.791566 13001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:26.798966 13001 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.125:38209
I20260925 00:43:26.799276 13068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.125:38209 every 8 connection(s)
I20260925 00:43:26.800004 13001 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
I20260925 00:43:26.802944 13069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:26.806790 13069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:43:26.809161 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 13001
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297006, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.125@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.72.125@KRBTEST.COM
I20260925 00:43:26.821399 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.808756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39302 (local address 127.6.72.126:35623)
0925 00:43:26.808910 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:26.808913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:26.810072 (+ 1159us) server_negotiation.cc:407] Connection header received
0925 00:43:26.811253 (+ 1181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:26.811256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:26.811304 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:26.811372 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:26.812335 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.812856 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.814031 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.814339 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.814443 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.817470 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:26.817475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:26.817478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:26.817508 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:26.819191 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.819731 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.819735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.819737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.819820 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.820180 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.820184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.820186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.820511 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:26.820610 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:26.821116 (+ 506us) server_negotiation.cc:299] Negotiation successful
0925 00:43:26.821236 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:43:26.822220 13071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.809273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:39302)
0925 00:43:26.809904 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:26.809964 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:26.811010 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:26.811449 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:26.811460 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:26.811894 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:26.812141 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.812155 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.812984 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.812988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.813902 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.813913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.814499 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.814503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.814730 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.815385 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:26.815406 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:26.817268 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:26.819317 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.819325 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.819335 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.819627 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.819923 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.819926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.819928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.820066 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.820593 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:26.820600 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:26.820871 (+ 271us) client_negotiation.cc:769] Sending connection context
0925 00:43:26.821060 (+ 189us) client_negotiation.cc:240] Negotiation successful
0925 00:43:26.821379 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":28,"thread_start_us":167,"threads_started":1}
I20260925 00:43:26.823210 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.810365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58246 (local address 127.6.72.125:38209)
0925 00:43:26.810650 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:26.810662 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:26.810704 (+ 42us) server_negotiation.cc:407] Connection header received
0925 00:43:26.811010 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:26.811032 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:26.811327 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:26.811516 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:26.812230 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.813279 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.814201 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.814722 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.814832 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.817873 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:26.817880 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:26.817893 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:26.817916 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:26.819839 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.820359 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.820365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.820366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.820453 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.820731 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.820735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.820737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.821048 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:26.821256 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:26.822234 (+ 978us) server_negotiation.cc:299] Negotiation successful
0925 00:43:26.823000 (+ 766us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":197,"spinlock_wait_cycles":19328,"thread_start_us":101,"threads_started":1}
I20260925 00:43:26.824023 13069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297006, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.125@KRBTEST.COM for oryx/127.6.72.125@KRBTEST.COM
I20260925 00:43:26.836166 13071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.824800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:38209 (local address 127.0.0.1:58256)
0925 00:43:26.824921 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:26.824935 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:26.825008 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:26.828666 (+ 3658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:26.828669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:26.828821 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:26.828942 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.828949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.829739 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.829742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.830287 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.830294 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.831253 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.831256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.831347 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.831875 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:26.831886 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:26.833256 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:26.834846 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.834848 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.834850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.835039 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.835320 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.835322 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.835324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.835394 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.835744 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:26.835746 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:26.835819 (+ 73us) client_negotiation.cc:769] Sending connection context
0925 00:43:26.835910 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 00:43:26.836006 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 00:43:26.836347 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.824850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.6.72.125:38209)
0925 00:43:26.828421 (+ 3571us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:26.828425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:26.828439 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:26.828475 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:26.828477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:26.828525 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:26.828607 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:26.829112 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.829614 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.830411 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.831098 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.831223 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.833384 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:26.833388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:26.833390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:26.833411 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:26.834743 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.835149 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.835152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.835153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.835227 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.835480 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.835482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.835483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.835659 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:26.835733 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:26.836006 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 00:43:26.836102 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3471}
I20260925 00:43:26.839484 13069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: Bootstrap starting.
I20260925 00:43:26.842078 13069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:26.842747 13069 log.cc:824] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:26.844831 13069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: No bootstrap required, opened a new log
I20260925 00:43:26.847713 13069 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } }
I20260925 00:43:26.847911 13069 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:26.847949 13069 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1af392529814d1cb61dca0eca097f5e, State: Initialized, Role: FOLLOWER
I20260925 00:43:26.848431 13069 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } }
I20260925 00:43:26.848874 13075 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } } }
I20260925 00:43:26.849015 13075 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:43:26.849620 13069 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: configured and running, proceeding with master startup.
W20260925 00:43:26.854130 13086 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:43:26.854220 13086 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:43:26.856666 13001 master_runner.cc:428] Detected that this master a1af392529814d1cb61dca0eca097f5e is joining an existing cluster
I20260925 00:43:26.856717 13001 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.72.125:38209
I20260925 00:43:26.868209 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.858868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39304 (local address 127.6.72.126:35623)
0925 00:43:26.859016 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:26.859022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:26.859783 (+ 761us) server_negotiation.cc:407] Connection header received
0925 00:43:26.859884 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:26.859890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:26.859950 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:26.860061 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:26.860689 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.861576 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.862533 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.862797 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.862892 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.864516 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:26.864520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:26.864522 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:26.864549 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:26.866248 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.866738 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.866743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.866745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.866832 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.867116 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.867121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.867123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.867315 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:26.867432 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:26.867920 (+ 488us) server_negotiation.cc:299] Negotiation successful
0925 00:43:26.868031 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:43:26.868259 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.858815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:39304)
0925 00:43:26.859666 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:26.859684 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:26.859792 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:26.860073 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:26.860075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:26.860394 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:26.860566 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.860573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.861697 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.861701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.862417 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.862425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.862918 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.862921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.863063 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.863828 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:26.863845 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:26.864359 (+ 514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:26.866377 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.866383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.866385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.866632 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.866931 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.866934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.866936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.867028 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.867700 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:26.867701 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:26.867763 (+ 62us) client_negotiation.cc:769] Sending connection context
0925 00:43:26.867866 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 00:43:26.868021 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":93,"spinlock_wait_cycles":4608,"thread_start_us":93,"threads_started":1}
I20260925 00:43:26.868279 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.859352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58262 (local address 127.6.72.125:38209)
0925 00:43:26.859488 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:26.859494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:26.859806 (+ 312us) server_negotiation.cc:407] Connection header received
0925 00:43:26.859939 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:26.859943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:26.859978 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:26.860043 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:26.860778 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.861551 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.862770 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.863085 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.863191 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.864652 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:26.864655 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:26.864658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:26.864679 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:26.866604 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.867063 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.867069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.867071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.867138 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.867415 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.867419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.867421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.867595 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:26.867667 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:26.867969 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 00:43:26.868075 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":55}
I20260925 00:43:26.869289 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.859365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:38209 (local address 127.0.0.1:58262)
0925 00:43:26.859710 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:26.859720 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:26.859864 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:26.860146 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:26.860148 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:26.860440 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:26.860613 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.860621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.861677 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.861680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.862655 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.862665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.863191 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.863194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.863296 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.864060 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:26.864071 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:26.864512 (+ 441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:26.866729 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.866732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.866734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.866970 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.867231 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.867234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.867235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.867319 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.867687 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:26.867690 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:26.867789 (+ 99us) client_negotiation.cc:769] Sending connection context
0925 00:43:26.868991 (+ 1202us) client_negotiation.cc:240] Negotiation successful
0925 00:43:26.869117 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":48,"spinlock_wait_cycles":2688,"thread_start_us":98,"threads_started":1}
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297005, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.126@KRBTEST.COM for oryx/127.6.72.125@KRBTEST.COM
I20260925 00:43:26.882997 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.872866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:38209 (local address 127.0.0.1:58276)
0925 00:43:26.873236 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:26.873284 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:26.873415 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:26.873779 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:26.873785 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:26.874129 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:26.874284 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.874290 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.875293 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.875295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.875985 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.875996 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.876857 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.876863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.876991 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.877569 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:26.877581 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:26.878991 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:26.881011 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.881024 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.881033 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.881363 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.881718 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.881724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.881728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.881838 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.882293 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:26.882300 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:26.882459 (+ 159us) client_negotiation.cc:769] Sending connection context
0925 00:43:26.882634 (+ 175us) client_negotiation.cc:240] Negotiation successful
0925 00:43:26.882858 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":107,"threads_started":1}
I20260925 00:43:26.883572 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.872908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58276 (local address 127.6.72.125:38209)
0925 00:43:26.873068 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:26.873073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:26.873364 (+ 291us) server_negotiation.cc:407] Connection header received
0925 00:43:26.873545 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:26.873549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:26.873600 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:26.873693 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:26.874407 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.875183 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.876281 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.876615 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.876722 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.879124 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:26.879128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:26.879132 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:26.879157 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:26.880892 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.881469 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.881473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.881475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.881557 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.881944 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.881947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.881949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.882182 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:26.882279 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:26.883285 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0925 00:43:26.883405 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260925 00:43:26.884948 12537 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.72.125:38209: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a1af392529814d1cb61dca0eca097f5e"
member_type: NON_VOTER
last_known_addr {
host: "127.6.72.125"
port: 38209
}
attrs {
promote: true
}
}
dest_uuid: "4c635738475c448680441d391652e4c5"
cas_config_opid_index: -1
I20260925 00:43:26.885686 12537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.6.72.124@KRBTEST.COM; defaulting to no policy
I20260925 00:43:26.887410 13097 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a1af392529814d1cb61dca0eca097f5e (127.6.72.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } } }
I20260925 00:43:26.888379 13043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:26.889791 13097 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.72.125:38209
I20260925 00:43:26.890007 13043 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 FOLLOWER]: Refusing update from remote peer 4c635738475c448680441d391652e4c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 00:43:26.890012 13097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } } }
I20260925 00:43:26.890103 13097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:26.890261 13001 master.cc:537] Master@127.6.72.125:38209 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.124@KRBTEST.COM" created.
I20260925 00:43:26.890689 13097 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 00:43:26.892382 12523 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260925 00:43:26.894793 12523 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 -> Peer a1af392529814d1cb61dca0eca097f5e (127.6.72.125:38209): Couldn't send request to peer a1af392529814d1cb61dca0eca097f5e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 00:43:26.895777 13001 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:26.896065 13001 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:26.896183 13001 tablet_replica.cc:330] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: stopping tablet replica
I20260925 00:43:26.900667 13001 master.cc:552] Master@127.6.72.125:38209 shutdown complete.
I20260925 00:43:26.900739 13001 master_runner.cc:305] Clearing existing system tablet
I20260925 00:43:26.900969 13001 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:26.901041 13001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:26.902130 13001 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 00:43:26.903074 13102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.903275 13001 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:26.903390 13001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:26.903424 13001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal
uuid: "a1af392529814d1cb61dca0eca097f5e"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "21aa4fa2e7e9ca172e30b32eadeec689"
server_key_iv: "c2bf71ce29d52de71547cee18a721354"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.905746 13001 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:26.908927 13001 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 00:43:26.909022 13001 log.cc:1196] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 00:43:26.909271 13001 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: Deleting consensus metadata
I20260925 00:43:26.910050 13001 master_runner.cc:315] Copying system tablet from 127.6.72.126:35623
I20260925 00:43:26.911278 13001 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: tablet copy: Beginning tablet copy session from remote peer at address 127.6.72.126:35623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.124.keytab.
Entry for principal oryx/127.6.72.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.124.keytab.
I20260925 00:43:26.912461 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.124:36087
--webserver_interface=127.6.72.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.124.keytab
--principal=oryx/127.6.72.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:35623,127.6.72.125:38209,127.6.72.124:36087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:43:26.920926 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.912006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39310 (local address 127.6.72.126:35623)
0925 00:43:26.912220 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:26.912226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:26.912379 (+ 153us) server_negotiation.cc:407] Connection header received
0925 00:43:26.912590 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:26.912596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:26.912670 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:26.912734 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:26.913341 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.914024 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.914932 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.915168 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.915290 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.916814 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:26.916818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:26.916821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:26.916852 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:26.918762 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.919340 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.919345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.919347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.919438 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:26.919796 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:26.919799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:26.919801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:26.920019 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:26.920118 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:26.920602 (+ 484us) server_negotiation.cc:299] Negotiation successful
0925 00:43:26.920737 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 00:43:26.921572 13108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:26.911924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:39310)
0925 00:43:26.912289 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:26.912308 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:26.912445 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:26.912899 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:26.912901 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:26.913090 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:26.913204 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.913210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.914152 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.914158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.914785 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:26.914794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:26.915292 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:26.915295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:26.915407 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:26.916043 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:26.916065 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:26.916630 (+ 565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:26.918894 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.918901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.918905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.919218 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.919556 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:26.919562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:26.919567 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:26.919682 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:26.920132 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:26.920138 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:26.920374 (+ 236us) client_negotiation.cc:769] Sending connection context
0925 00:43:26.920528 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 00:43:26.920724 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":101,"threads_started":1}
I20260925 00:43:26.922309 12557 tablet_copy_service.cc:140] P 4c635738475c448680441d391652e4c5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a1af392529814d1cb61dca0eca097f5e ({username='oryx', principal='oryx/127.6.72.125@KRBTEST.COM'} at 127.0.0.1:39310)
I20260925 00:43:26.922562 12557 tablet_copy_service.cc:161] P 4c635738475c448680441d391652e4c5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a1af392529814d1cb61dca0eca097f5e at {username='oryx', principal='oryx/127.6.72.125@KRBTEST.COM'} at 127.0.0.1:39310: session id = a1af392529814d1cb61dca0eca097f5e-00000000000000000000000000000000
I20260925 00:43:26.925077 12557 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 00:43:26.927080 13001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 00:43:26.929555 13001 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: tablet copy: Starting download of 0 data blocks...
I20260925 00:43:26.929785 13001 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: tablet copy: Starting download of 1 WAL segments...
I20260925 00:43:26.931836 13001 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 00:43:26.933949 12557 tablet_copy_service.cc:342] P 4c635738475c448680441d391652e4c5: Request end of tablet copy session a1af392529814d1cb61dca0eca097f5e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.72.125@KRBTEST.COM'} at 127.0.0.1:39310
I20260925 00:43:26.934113 12557 tablet_copy_service.cc:433] P 4c635738475c448680441d391652e4c5: ending tablet copy session a1af392529814d1cb61dca0eca097f5e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a1af392529814d1cb61dca0eca097f5e
I20260925 00:43:26.942286 13001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:26.944199 13113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:26.944331 13110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:26.944612 13001 server_base.cc:1034] running on GCE node
W20260925 00:43:26.944850 13111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:26.945067 13001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:26.945379 13001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:26.946532 13001 hybrid_clock.cc:648] HybridClock initialized: now 1790297006946504 us; error 43 us; skew 500 ppm
Sep 25 00:43:26 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297006, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:26.948310 13001 init.cc:377] Logged in from keytab as oryx/127.6.72.125@KRBTEST.COM (short username oryx)
I20260925 00:43:26.948846 13001 webserver.cc:460] Webserver started at http://127.6.72.125:35991/ using document root <none> and password file <none>
I20260925 00:43:26.949065 13001 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:26.949116 13001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:26.950603 13001 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 00:43:26.951704 13120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:26.951964 13001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:26.952044 13001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal
uuid: "a1af392529814d1cb61dca0eca097f5e"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "21aa4fa2e7e9ca172e30b32eadeec689"
server_key_iv: "c2bf71ce29d52de71547cee18a721354"
server_key_version: "encryptionkey@0"
I20260925 00:43:26.952126 13001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:26.960142 13001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:26.962604 13001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:26.967670 13001 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.125:38209
I20260925 00:43:26.967953 13172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.125:38209 every 8 connection(s)
I20260925 00:43:26.968336 13001 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
I20260925 00:43:26.970080 13173 sys_catalog.cc:263] Verifying existing consensus state
I20260925 00:43:26.971060 13173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: Bootstrap starting.
I20260925 00:43:26.981338 13173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:43:26.981694 13173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: Bootstrap complete.
I20260925 00:43:26.982311 13173 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } }
I20260925 00:43:26.982452 13173 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a1af392529814d1cb61dca0eca097f5e, State: Initialized, Role: LEARNER
I20260925 00:43:26.982577 13173 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } }
I20260925 00:43:26.983057 13173 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:26.983296 13176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } } }
I20260925 00:43:26.983525 13176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: This master's current role is: LEARNER
I20260925 00:43:26.986454 13187 catalog_manager.cc:1259] Loaded cluster ID: 497d8ee0601248caa0af71534968b68a
I20260925 00:43:26.986558 13187 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: loading cluster ID for follower catalog manager: success
I20260925 00:43:26.988694 13187 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: acquiring CA information for follower catalog manager: success
I20260925 00:43:26.990022 13187 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 00:43:27.050920 13109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:27.051192 13109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:27.051234 13109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:27.055598 13109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:27.055675 13109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:27.055696 13109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:27.055712 13109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:27.055728 13109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:27.061517 13109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:35623,127.6.72.125:38209,127.6.72.124:36087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.124.keytab
--principal=oryx/127.6.72.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.124:36087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.72.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:27.062696 13109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:27.063745 13109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:27.072340 13195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:27.072342 13193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:27.072340 13192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:27.072600 13109 server_base.cc:1034] running on GCE node
I20260925 00:43:27.073190 13109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:27.074361 13109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:27.075559 13109 hybrid_clock.cc:648] HybridClock initialized: now 1790297007075517 us; error 46 us; skew 500 ppm
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:27.078472 13109 init.cc:377] Logged in from keytab as oryx/127.6.72.124@KRBTEST.COM (short username oryx)
I20260925 00:43:27.079370 13109 webserver.cc:460] Webserver started at http://127.6.72.124:46433/ using document root <none> and password file <none>
I20260925 00:43:27.079948 13109 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:27.080053 13109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:27.080348 13109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:27.082509 13109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data/instance:
uuid: "3b718e270acb433babe5435e4ba571b4"
format_stamp: "Formatted at 2026-09-25 00:43:27 on dist-test-slave-w8nh"
server_key: "ca5d41160dae7af28f97c82c1485ca9a"
server_key_iv: "7cdeb920e2cfa06f304f8fb43cfca2dc"
server_key_version: "encryptionkey@0"
I20260925 00:43:27.083117 13109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal/instance:
uuid: "3b718e270acb433babe5435e4ba571b4"
format_stamp: "Formatted at 2026-09-25 00:43:27 on dist-test-slave-w8nh"
server_key: "ca5d41160dae7af28f97c82c1485ca9a"
server_key_iv: "7cdeb920e2cfa06f304f8fb43cfca2dc"
server_key_version: "encryptionkey@0"
I20260925 00:43:27.087323 13109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 00:43:27.090245 13202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.091599 13109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:27.091766 13109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal
uuid: "3b718e270acb433babe5435e4ba571b4"
format_stamp: "Formatted at 2026-09-25 00:43:27 on dist-test-slave-w8nh"
server_key: "ca5d41160dae7af28f97c82c1485ca9a"
server_key_iv: "7cdeb920e2cfa06f304f8fb43cfca2dc"
server_key_version: "encryptionkey@0"
I20260925 00:43:27.091902 13109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:27.123327 13109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:27.126935 13109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:27.127166 13109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:27.134099 12728 heartbeater.cc:503] Master 127.6.72.126:35623 was elected leader, sending a full tablet report...
I20260925 00:43:27.136471 13109 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.124:36087
I20260925 00:43:27.136730 13254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.124:36087 every 8 connection(s)
I20260925 00:43:27.137530 13109 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
I20260925 00:43:27.141134 13255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:27.141121 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 13109
I20260925 00:43:27.146833 13255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.72.124@KRBTEST.COM
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.124@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:27.157985 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.142854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.6.72.124:36087)
0925 00:43:27.143379 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.143398 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.143434 (+ 36us) server_negotiation.cc:407] Connection header received
0925 00:43:27.144361 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.144380 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.144724 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.145000 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.146178 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.147216 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.148592 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.149373 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.149533 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.152731 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.152737 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.152747 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.152777 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.154537 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.155003 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.155009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.155016 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.155181 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.155481 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.155486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.155488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.156022 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.156319 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.156550 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.156882 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"spinlock_wait_cycles":768,"thread_start_us":187,"threads_started":1}
I20260925 00:43:27.159608 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.148971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:39316)
0925 00:43:27.149983 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.150021 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.150162 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.150505 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.150510 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.150880 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.151001 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.151007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.152082 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.152085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.152947 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.152957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.153879 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.153882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.153973 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.154477 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.154517 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.156061 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.157855 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.157861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.157866 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.158143 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.158499 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.158502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.158504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.158592 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.158969 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.158974 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.159093 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.159308 (+ 215us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.159465 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"spinlock_wait_cycles":8064,"thread_start_us":209,"threads_started":1}
I20260925 00:43:27.159648 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.149276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39316 (local address 127.6.72.126:35623)
0925 00:43:27.149415 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.149419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.150108 (+ 689us) server_negotiation.cc:407] Connection header received
0925 00:43:27.150296 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.150300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.150357 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.150447 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.151132 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.151968 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.153327 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.153577 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.153727 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.156201 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.156205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.156208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.156294 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.157751 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.158304 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.158311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.158314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.158402 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.158692 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.158694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.158695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.158871 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.158945 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.159384 (+ 439us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.159504 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 00:43:27.160794 13255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.124@KRBTEST.COM for oryx/127.6.72.125@KRBTEST.COM
I20260925 00:43:27.169523 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.161355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:38209 (local address 127.0.0.1:58282)
0925 00:43:27.161478 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.161497 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.161564 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.162011 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.162014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.162189 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.162282 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.162287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.163189 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.163196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.163755 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.163763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.164226 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.164230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.164355 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.164816 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.164827 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.166223 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.167943 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.167948 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.167950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.168152 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.168462 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.168466 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.168468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.168549 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.168979 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.168982 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.169081 (+ 99us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.169192 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.169338 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 00:43:27.169533 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.161539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58282 (local address 127.6.72.125:38209)
0925 00:43:27.161805 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.161808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.161820 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:43:27.161855 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.161857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.161895 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.161997 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.162397 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.163080 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.163876 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.164114 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.164229 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.166372 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.166376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.166381 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.166411 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.167827 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.168279 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.168282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.168284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.168366 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.168649 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.168652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.168654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.168885 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.168990 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.169255 (+ 265us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.169374 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":101,"threads_started":1}
I20260925 00:43:27.170423 13255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.124@KRBTEST.COM for oryx/127.6.72.124@KRBTEST.COM
I20260925 00:43:27.179311 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.171012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40710 (local address 127.6.72.124:36087)
0925 00:43:27.171142 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.171148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.171232 (+ 84us) server_negotiation.cc:407] Connection header received
0925 00:43:27.171404 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.171408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.171457 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.171570 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.171978 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.173026 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.173872 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.174142 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.174272 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.176437 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.176442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.176446 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.176470 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.177723 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.178182 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.178185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.178186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.178251 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.178521 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.178524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.178526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.178694 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.178793 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.179030 (+ 237us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.179175 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":49}
I20260925 00:43:27.179311 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.170938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:36087 (local address 127.0.0.1:40710)
0925 00:43:27.171140 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.171159 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.171264 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.171578 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.171582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.171782 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.171853 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.171858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.173178 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.173182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.173742 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.173750 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.174268 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.174272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.174440 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.174889 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.174903 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.176316 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.177859 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.177864 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.177866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.178053 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.178347 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.178350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.178352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.178431 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.178792 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.178795 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.178873 (+ 78us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.178978 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.179116 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":44}
I20260925 00:43:27.182381 13255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Bootstrap starting.
I20260925 00:43:27.185624 13255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:27.186436 13255 log.cc:824] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:27.188509 13255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: No bootstrap required, opened a new log
I20260925 00:43:27.191538 13255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } }
I20260925 00:43:27.191771 13255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:27.191807 13255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b718e270acb433babe5435e4ba571b4, State: Initialized, Role: FOLLOWER
I20260925 00:43:27.192355 13255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } }
I20260925 00:43:27.192883 13261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } } }
I20260925 00:43:27.193037 13261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:43:27.193602 13255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 00:43:27.198822 13272 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:43:27.198938 13272 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:43:27.202280 13109 master_runner.cc:428] Detected that this master 3b718e270acb433babe5435e4ba571b4 is joining an existing cluster
I20260925 00:43:27.202359 13109 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.72.124:36087
I20260925 00:43:27.211817 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 12599
I20260925 00:43:27.214128 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.204886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40718 (local address 127.6.72.124:36087)
0925 00:43:27.204998 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.205003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.205512 (+ 509us) server_negotiation.cc:407] Connection header received
0925 00:43:27.205653 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.205655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.205692 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.205737 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.206440 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.207202 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.208412 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.208745 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.208815 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.210382 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.210385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.210386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.210403 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.212214 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.212719 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.212725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.212730 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.212805 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.213097 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.213102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.213104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.213332 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.213434 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.213743 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.213938 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":46}
I20260925 00:43:27.214178 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.205070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:36087 (local address 127.0.0.1:40718)
0925 00:43:27.205436 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.205449 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.205564 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.205854 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.205857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.206140 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.206324 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.206332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.207332 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.207335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.208295 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.208305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.208963 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.208967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.209052 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.209796 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.209811 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.210253 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.212353 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.212356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.212358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.212619 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.212903 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.212909 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.212914 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.212995 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.213439 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.213444 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.213558 (+ 114us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.213677 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.213849 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":4,"thread_start_us":126,"threads_started":1}
I20260925 00:43:27.219305 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:45781
--local_ip_for_outbound_sockets=127.6.72.65
--tserver_master_addrs=127.6.72.126:35623
--webserver_port=35993
--webserver_interface=127.6.72.65
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab
--principal=oryx/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.72.126:35623,127.6.72.125:38209
--tserver_master_addrs=127.6.72.126:35623,127.6.72.125:38209,127.6.72.124:36087 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:43:27.239219 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.204855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39332 (local address 127.6.72.126:35623)
0925 00:43:27.204984 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.204989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.205237 (+ 248us) server_negotiation.cc:407] Connection header received
0925 00:43:27.205451 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.205454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.205505 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.205560 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.206460 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.207279 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.208598 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.208932 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.209010 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.234465 (+ 25455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.234470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.234473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.234507 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.236807 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.237608 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.237612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.237614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.237709 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.238091 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.238095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.238097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.238347 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.238435 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.238891 (+ 456us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.239030 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:43:27.239219 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.205501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.6.72.125:38209)
0925 00:43:27.205633 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.205639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.205859 (+ 220us) server_negotiation.cc:407] Connection header received
0925 00:43:27.206031 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.206036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.206093 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.206145 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.206724 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.207464 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.208601 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.208944 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.209009 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.234464 (+ 25455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.234470 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.234473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.234507 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.236815 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.237597 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.237601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.237602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.237704 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.238095 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.238099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.238101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.238347 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.238435 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.238890 (+ 455us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.239043 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:43:27.239929 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.205206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:38209 (local address 127.0.0.1:58292)
0925 00:43:27.205782 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.205798 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.205917 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.206307 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.206310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.206493 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.206587 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.206593 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.207588 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.207591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.208487 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.208496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.209158 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.209161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.209579 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.233180 (+ 23601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.233223 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.234189 (+ 966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.236986 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.236990 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.236993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.237449 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.237855 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.237858 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.237860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.237971 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.238566 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.238569 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.238659 (+ 90us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.239382 (+ 723us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.239644 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":337,"spinlock_wait_cycles":32384,"thread_start_us":74,"threads_started":1}
I20260925 00:43:27.241120 13261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:27.241304 13261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } }
I20260925 00:43:27.242236 13261 leader_election.cc:288] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c635738475c448680441d391652e4c5 (127.6.72.126:35623), a1af392529814d1cb61dca0eca097f5e (127.6.72.125:38209)
I20260925 00:43:27.242694 13147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b718e270acb433babe5435e4ba571b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1af392529814d1cb61dca0eca097f5e" is_pre_election: true
I20260925 00:43:27.242889 13147 raft_consensus.cc:1799] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 LEARNER]: Handling vote request from an unknown peer 3b718e270acb433babe5435e4ba571b4
I20260925 00:43:27.243074 13147 raft_consensus.cc:2396] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 3b718e270acb433babe5435e4ba571b4 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260925 00:43:27.243485 12547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b718e270acb433babe5435e4ba571b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c635738475c448680441d391652e4c5" is_pre_election: true
I20260925 00:43:27.244158 13205 leader_election.cc:302] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3b718e270acb433babe5435e4ba571b4; no voters: 4c635738475c448680441d391652e4c5, a1af392529814d1cb61dca0eca097f5e
I20260925 00:43:27.244467 13261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:27.245965 13261 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 00:43:27.244392 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.204795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:39332)
0925 00:43:27.205160 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.205175 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.205328 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.205717 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.205720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.206101 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.206324 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.206332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.207406 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.207409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.208460 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.208470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.209158 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.209161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.209308 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.233179 (+ 23871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.233205 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.234189 (+ 984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.236986 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.236990 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.236993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.237449 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.237845 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.237848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.237850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.237962 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.238534 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.238538 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.238657 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.239954 (+ 1297us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.244110 (+ 4156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":90,"spinlock_wait_cycles":30336,"thread_start_us":91,"threads_started":1}
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297005, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.126@KRBTEST.COM for oryx/127.6.72.124@KRBTEST.COM
I20260925 00:43:27.259207 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.250098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:36087 (local address 127.0.0.1:40722)
0925 00:43:27.250280 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.250297 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.250395 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.250741 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.250744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.250955 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.251051 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.251057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.251900 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.251903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.252640 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.252651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.253222 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.253227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.253346 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.253910 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.253928 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.255528 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.257451 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.257456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.257459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.257688 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.257998 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.258003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.258005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.258104 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.258616 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.258619 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.258738 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.258859 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.259029 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260925 00:43:27.259244 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.250140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40722 (local address 127.6.72.124:36087)
0925 00:43:27.250270 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.250275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.250345 (+ 70us) server_negotiation.cc:407] Connection header received
0925 00:43:27.250545 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.250548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.250628 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.250722 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.251182 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.251734 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.252781 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.253097 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.253220 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.255679 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.255683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.255686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.255714 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.257320 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.257798 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.257803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.257806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.257897 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.258210 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.258214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.258216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.258414 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.258616 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.258939 (+ 323us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.259109 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 00:43:27.260634 12537 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.72.124:36087: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3b718e270acb433babe5435e4ba571b4"
member_type: NON_VOTER
last_known_addr {
host: "127.6.72.124"
port: 36087
}
attrs {
promote: true
}
}
dest_uuid: "4c635738475c448680441d391652e4c5"
cas_config_opid_index: 5
I20260925 00:43:27.261227 12537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } }
I20260925 00:43:27.262284 13097 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3b718e270acb433babe5435e4ba571b4 (127.6.72.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.263477 13097 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.72.124:36087
I20260925 00:43:27.263667 13097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.263778 13097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:27.264959 13109 master.cc:537] Master@127.6.72.124:36087 shutting down...
I20260925 00:43:27.265141 13228 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 FOLLOWER]: Refusing update from remote peer 4c635738475c448680441d391652e4c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 00:43:27.265904 13096 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 00:43:27.267778 12524 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 -> Peer 3b718e270acb433babe5435e4ba571b4 (127.6.72.124:36087): Couldn't send request to peer 3b718e270acb433babe5435e4ba571b4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 00:43:27.271292 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.264178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.6.72.125:38209)
0925 00:43:27.264331 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.264337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.265488 (+ 1151us) server_negotiation.cc:407] Connection header received
0925 00:43:27.265729 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.265732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.265817 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.265895 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:27.266322 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.267057 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.268871 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.269860 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.269939 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.270096 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.270750 (+ 654us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.271156 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 00:43:27.271709 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.264096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:38209 (local address 127.0.0.1:58296)
0925 00:43:27.265404 (+ 1308us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.265420 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.265587 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.266069 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.266072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.266090 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:27.266186 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.266194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.267186 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.267190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.268737 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.268748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.270194 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.270198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.270286 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.270633 (+ 347us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.271501 (+ 868us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.271592 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1226}
I20260925 00:43:27.272619 13109 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:27.272693 13147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 LEARNER]: Refusing update from remote peer 4c635738475c448680441d391652e4c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 00:43:27.272898 13109 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:27.273008 13109 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: stopping tablet replica
I20260925 00:43:27.273138 13096 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 00:43:27.275074 13147 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3b718e270acb433babe5435e4ba571b4 (127.6.72.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.277160 13283 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c635738475c448680441d391652e4c5. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.277374 13283 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: This master's current role is: LEARNER
I20260925 00:43:27.277177 13176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: NON_VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: true } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.277480 13176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: This master's current role is: LEARNER
I20260925 00:43:27.281730 13096 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5: attempting to promote NON_VOTER a1af392529814d1cb61dca0eca097f5e to VOTER
I20260925 00:43:27.282232 13096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } }
I20260925 00:43:27.283715 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.283441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:36087 (local address 127.0.0.1:40730)
0925 00:43:27.283539 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.283624 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:36087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 00:43:27.284091 13147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 LEARNER]: Refusing update from remote peer 4c635738475c448680441d391652e4c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 00:43:27.284912 13284 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 00:43:27.285131 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.284601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:36087 (local address 127.0.0.1:40744)
0925 00:43:27.284951 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.285003 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:36087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277}
W20260925 00:43:27.285270 12524 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 -> Peer 3b718e270acb433babe5435e4ba571b4 (127.6.72.124:36087): Couldn't send request to peer 3b718e270acb433babe5435e4ba571b4. Status: Network error: Client connection negotiation failed: client connection to 127.6.72.124:36087: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 00:43:27.286527 13097 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a1af392529814d1cb61dca0eca097f5e (127.6.72.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.287969 13097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.288089 13097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:27.288266 13097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.288364 13097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:27.289408 13109 master.cc:552] Master@127.6.72.124:36087 shutdown complete.
I20260925 00:43:27.289264 13147 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a1af392529814d1cb61dca0eca097f5e (127.6.72.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.289528 13109 master_runner.cc:305] Clearing existing system tablet
I20260925 00:43:27.289829 13109 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:27.289922 13109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:27.290990 13176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.291107 13176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:43:27.291561 13109 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:27.292407 13286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.292613 13109 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:27.292739 13109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 00:43:27.292773 13109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal
uuid: "3b718e270acb433babe5435e4ba571b4"
format_stamp: "Formatted at 2026-09-25 00:43:27 on dist-test-slave-w8nh"
server_key: "ca5d41160dae7af28f97c82c1485ca9a"
server_key_iv: "7cdeb920e2cfa06f304f8fb43cfca2dc"
server_key_version: "encryptionkey@0"
I20260925 00:43:27.294721 13109 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:27.296828 13109 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 00:43:27.296939 13109 log.cc:1196] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 00:43:27.297282 13109 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Deleting consensus metadata
I20260925 00:43:27.297493 13109 master_runner.cc:315] Copying system tablet from 127.6.72.126:35623
I20260925 00:43:27.298655 13109 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: tablet copy: Beginning tablet copy session from remote peer at address 127.6.72.126:35623
I20260925 00:43:27.312021 13292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.299210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:39344)
0925 00:43:27.304454 (+ 5244us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.304469 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.304633 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.304910 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.304913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.305171 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.305289 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.305296 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.306070 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.306075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.307057 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.307066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.307531 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.307534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.307630 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.308206 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.308217 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.308643 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.310479 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.310483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.310486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.310696 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.310974 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.310979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.310981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.311066 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.311456 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.311459 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.311562 (+ 103us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.311674 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.311826 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5160,"thread_start_us":87,"threads_started":1}
I20260925 00:43:27.312009 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.299289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.6.72.126:35623)
0925 00:43:27.299425 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.299430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.304572 (+ 5142us) server_negotiation.cc:407] Connection header received
0925 00:43:27.304754 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.304757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.304804 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.304890 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.305418 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.305955 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.307192 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.307412 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.307523 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.308802 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.308805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.308807 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.308831 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.310373 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.310796 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.310801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.310803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.310885 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.311175 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.311179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.311181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.311362 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.311468 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.311726 (+ 258us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.311833 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 00:43:27.312937 12557 tablet_copy_service.cc:140] P 4c635738475c448680441d391652e4c5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3b718e270acb433babe5435e4ba571b4 ({username='oryx', principal='oryx/127.6.72.124@KRBTEST.COM'} at 127.0.0.1:39344)
I20260925 00:43:27.313050 12557 tablet_copy_service.cc:161] P 4c635738475c448680441d391652e4c5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3b718e270acb433babe5435e4ba571b4 at {username='oryx', principal='oryx/127.6.72.124@KRBTEST.COM'} at 127.0.0.1:39344: session id = 3b718e270acb433babe5435e4ba571b4-00000000000000000000000000000000
I20260925 00:43:27.313898 12557 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 00:43:27.315160 13109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 00:43:27.317162 13109 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: tablet copy: Starting download of 0 data blocks...
I20260925 00:43:27.317375 13109 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: tablet copy: Starting download of 1 WAL segments...
I20260925 00:43:27.318859 13109 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 00:43:27.320520 12557 tablet_copy_service.cc:342] P 4c635738475c448680441d391652e4c5: Request end of tablet copy session 3b718e270acb433babe5435e4ba571b4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.72.124@KRBTEST.COM'} at 127.0.0.1:39344
I20260925 00:43:27.320631 12557 tablet_copy_service.cc:433] P 4c635738475c448680441d391652e4c5: ending tablet copy session 3b718e270acb433babe5435e4ba571b4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3b718e270acb433babe5435e4ba571b4
I20260925 00:43:27.327683 13109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:27.329450 13293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:27.329526 13296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:27.329823 13109 server_base.cc:1034] running on GCE node
W20260925 00:43:27.330277 13294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:27.330508 13109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:27.330868 13109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:27.332017 13109 hybrid_clock.cc:648] HybridClock initialized: now 1790297007332025 us; error 42 us; skew 500 ppm
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:27.333531 13109 init.cc:377] Logged in from keytab as oryx/127.6.72.124@KRBTEST.COM (short username oryx)
I20260925 00:43:27.334029 13109 webserver.cc:460] Webserver started at http://127.6.72.124:35849/ using document root <none> and password file <none>
I20260925 00:43:27.334239 13109 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:27.334295 13109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:27.335446 13109 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 00:43:27.336410 13303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.336658 13109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 00:43:27.336731 13109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal
uuid: "3b718e270acb433babe5435e4ba571b4"
format_stamp: "Formatted at 2026-09-25 00:43:27 on dist-test-slave-w8nh"
server_key: "ca5d41160dae7af28f97c82c1485ca9a"
server_key_iv: "7cdeb920e2cfa06f304f8fb43cfca2dc"
server_key_version: "encryptionkey@0"
I20260925 00:43:27.336819 13109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:27.354511 13109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:27.356693 13109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:27.361905 13109 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.124:36087
I20260925 00:43:27.361917 13355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.124:36087 every 8 connection(s)
I20260925 00:43:27.362479 13109 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
I20260925 00:43:27.363883 13356 sys_catalog.cc:263] Verifying existing consensus state
I20260925 00:43:27.364817 13356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Bootstrap starting.
I20260925 00:43:27.366004 12862 heartbeater.cc:503] Master 127.6.72.126:35623 was elected leader, sending a full tablet report...
I20260925 00:43:27.374291 13356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:43:27.374657 13356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Bootstrap complete.
I20260925 00:43:27.375247 13356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } }
I20260925 00:43:27.375387 13356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b718e270acb433babe5435e4ba571b4, State: Initialized, Role: LEARNER
I20260925 00:43:27.375502 13356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } }
I20260925 00:43:27.375866 13362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.376030 13356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:27.376037 13362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: This master's current role is: LEARNER
I20260925 00:43:27.379869 13373 catalog_manager.cc:1259] Loaded cluster ID: 497d8ee0601248caa0af71534968b68a
I20260925 00:43:27.379952 13373 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: loading cluster ID for follower catalog manager: success
I20260925 00:43:27.381613 13373 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: acquiring CA information for follower catalog manager: success
W20260925 00:43:27.382387 13281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:27.382773 13281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:27.382853 13281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 00:43:27.383038 13373 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 00:43:27.389161 13281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:27.389245 13281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:27.389336 13281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:27.397651 13281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab
--principal=oryx/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:45781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=35993
--tserver_master_addrs=127.6.72.126:35623,127.6.72.125:38209,127.6.72.124:36087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:27.399115 13281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:27.400544 13281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:27.413285 13376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:27.413409 13378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:27.413283 13375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:27.413733 13281 server_base.cc:1034] running on GCE node
I20260925 00:43:27.414227 13281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:27.415010 13281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:27.416220 13281 hybrid_clock.cc:648] HybridClock initialized: now 1790297007416187 us; error 49 us; skew 500 ppm
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:27.419339 13281 init.cc:377] Logged in from keytab as oryx/127.6.72.65@KRBTEST.COM (short username oryx)
I20260925 00:43:27.420396 13281 webserver.cc:460] Webserver started at http://127.6.72.65:35993/ using document root <none> and password file <none>
I20260925 00:43:27.420989 13281 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:27.421067 13281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:27.425242 13281 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260925 00:43:27.427431 13385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.428998 13281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 00:43:27.429119 13281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "51d08a7b3a834c94982a8cb00ee067e9"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "edcfb5207ce0f9d7509e7bf6aedd5fd5"
server_key_iv: "c373090106071015fb9bc4c4efc96e20"
server_key_version: "encryptionkey@0"
I20260925 00:43:27.429742 13281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:27.446532 13281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:27.450505 13281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:27.450757 13281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:27.451449 13281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:27.452616 13281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:27.452721 13281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.452785 13281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:27.452845 13281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.463445 13281 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:45781
I20260925 00:43:27.463488 13498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:45781 every 8 connection(s)
I20260925 00:43:27.464658 13281 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:27.470376 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 13281
I20260925 00:43:27.470587 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 12733
I20260925 00:43:27.478101 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:33187
--local_ip_for_outbound_sockets=127.6.72.66
--tserver_master_addrs=127.6.72.126:35623
--webserver_port=33817
--webserver_interface=127.6.72.66
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab
--principal=oryx/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.72.126:35623,127.6.72.125:38209
--tserver_master_addrs=127.6.72.126:35623,127.6.72.125:38209,127.6.72.124:36087 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.125@KRBTEST.COM
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.124@KRBTEST.COM
I20260925 00:43:27.483948 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.469198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:39501 (local address 127.6.72.125:38209)
0925 00:43:27.469331 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.469336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.469360 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:43:27.469972 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.469976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.470035 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.470119 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.472231 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.473092 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.474393 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.474776 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.474852 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.479049 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.479057 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.479061 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.479112 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.481118 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.481812 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.481819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.481823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.481913 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.482343 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.482349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.482354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.482612 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.482693 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.483522 (+ 829us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.483677 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 00:43:27.485252 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.466667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:43877 (local address 127.6.72.126:35623)
0925 00:43:27.466819 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.466824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.468376 (+ 1552us) server_negotiation.cc:407] Connection header received
0925 00:43:27.469808 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.469812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.469872 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.469955 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.471639 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.472374 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.474361 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.474744 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.474836 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.479951 (+ 5115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.479959 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.479964 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.480000 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.482559 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.483483 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.483488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.483490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.483601 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.484025 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.484030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.484034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.484357 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.484441 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.484962 (+ 521us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.485097 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 00:43:27.486130 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.467519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:36087 (local address 127.6.72.65:54685)
0925 00:43:27.468180 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.468265 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.469395 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.470013 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.470023 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.470658 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.472897 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.472904 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.474243 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.474247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.475100 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.475109 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.475858 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.475861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.475973 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.476584 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.476598 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.480046 (+ 3448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.482687 (+ 2641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.482690 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.482692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.483044 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.483400 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.483407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.483411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.483503 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.484224 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.484227 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.484354 (+ 127us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.484466 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.484935 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":50,"spinlock_wait_cycles":62592,"thread_start_us":159,"threads_started":1}
I20260925 00:43:27.486146 13504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.467519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.65:43877)
0925 00:43:27.468180 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.468265 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.469395 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.470014 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.470023 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.470660 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.471427 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.471451 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.472566 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.472570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.474195 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.474208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.475155 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.475158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.475363 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.476433 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.476460 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.479774 (+ 3314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.482697 (+ 2923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.482700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.482702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.483359 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.483765 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.483768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.483770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.483904 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.484536 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.484538 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.484637 (+ 99us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.485404 (+ 767us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.485548 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":54,"spinlock_wait_cycles":7296,"thread_start_us":181,"threads_started":1}
I20260925 00:43:27.486987 13506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.468874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:38209 (local address 127.6.72.65:39501)
0925 00:43:27.469178 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.469192 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.469416 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.471779 (+ 2363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.471783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.471963 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.472083 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.472089 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.473241 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.473244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.474252 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.474264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.475085 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.475093 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.475364 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.476768 (+ 1404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.476779 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.478796 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.481278 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.481287 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.481313 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.481693 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.482028 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.482034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.482039 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.482236 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.482820 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.482832 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.483210 (+ 378us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.484557 (+ 1347us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.485013 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":285,"spinlock_wait_cycles":1792,"thread_start_us":97,"threads_started":1}
I20260925 00:43:27.487001 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.466915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:54685 (local address 127.6.72.124:36087)
0925 00:43:27.467153 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.467157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.468373 (+ 1216us) server_negotiation.cc:407] Connection header received
0925 00:43:27.469641 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.469644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.469711 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.469810 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.473039 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.474090 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.475249 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.475613 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.475689 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.480370 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.480378 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.480382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.480418 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.482505 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.483166 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.483173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.483177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.483275 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.483623 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.483629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.483634 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.484104 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.486564 (+ 2460us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.486657 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.486814 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":99,"threads_started":1}
I20260925 00:43:27.487649 13500 heartbeater.cc:347] Connected to a master server at 127.6.72.126:35623
I20260925 00:43:27.487933 13500 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:27.487972 13499 heartbeater.cc:347] Connected to a master server at 127.6.72.124:36087
I20260925 00:43:27.488059 13499 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:27.488495 13500 heartbeater.cc:511] Master 127.6.72.126:35623 requested a full tablet report, sending...
I20260925 00:43:27.488502 13499 heartbeater.cc:511] Master 127.6.72.124:36087 requested a full tablet report, sending...
I20260925 00:43:27.489173 13501 heartbeater.cc:347] Connected to a master server at 127.6.72.125:38209
I20260925 00:43:27.489246 13501 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:27.489423 13501 heartbeater.cc:511] Master 127.6.72.125:38209 requested a full tablet report, sending...
I20260925 00:43:27.490013 12537 ts_manager.cc:194] Re-registered known tserver with Master: 51d08a7b3a834c94982a8cb00ee067e9 (127.6.72.65:45781)
I20260925 00:43:27.490617 12537 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:43877
I20260925 00:43:27.490696 13137 ts_manager.cc:194] Registered new tserver with Master: 51d08a7b3a834c94982a8cb00ee067e9 (127.6.72.65:45781)
I20260925 00:43:27.491648 13312 ts_manager.cc:194] Registered new tserver with Master: 51d08a7b3a834c94982a8cb00ee067e9 (127.6.72.65:45781)
I20260925 00:43:27.581540 12996 heartbeater.cc:503] Master 127.6.72.126:35623 was elected leader, sending a full tablet report...
W20260925 00:43:27.598886 13507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:27.599185 13507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:27.599231 13507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:27.603541 13507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:27.603624 13507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:27.603694 13507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:27.609582 13507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab
--principal=oryx/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:33187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=33817
--tserver_master_addrs=127.6.72.126:35623,127.6.72.125:38209,127.6.72.124:36087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:27.610721 13507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:27.611778 13507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:27.621567 13513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:27.621661 13507 server_base.cc:1034] running on GCE node
W20260925 00:43:27.621567 13515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:27.621557 13512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:27.622247 13507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:27.622906 13507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:27.624121 13507 hybrid_clock.cc:648] HybridClock initialized: now 1790297007624090 us; error 36 us; skew 500 ppm
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:27.626935 13507 init.cc:377] Logged in from keytab as oryx/127.6.72.66@KRBTEST.COM (short username oryx)
I20260925 00:43:27.627897 13507 webserver.cc:460] Webserver started at http://127.6.72.66:33817/ using document root <none> and password file <none>
I20260925 00:43:27.628526 13507 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:27.628604 13507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:27.632666 13507 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:43:27.634972 13522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.636487 13507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:27.636598 13507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "b569cd1d3ae043839c852ca947f15c50"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "10db122b29b0893d035a959d20c41b3f"
server_key_iv: "e680c39506af6d718e003a660563ac76"
server_key_version: "encryptionkey@0"
I20260925 00:43:27.637230 13507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:27.650624 13507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:27.653923 13507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:27.654155 13507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:27.654785 13507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:27.655689 13507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:27.655758 13507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.655828 13507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:27.655879 13507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.665709 13507 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:33187
I20260925 00:43:27.665748 13635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:33187 every 8 connection(s)
I20260925 00:43:27.666929 13507 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:27.676884 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 13507
I20260925 00:43:27.677090 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 12867
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.124@KRBTEST.COM
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.125@KRBTEST.COM
I20260925 00:43:27.684988 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:46823
--local_ip_for_outbound_sockets=127.6.72.67
--tserver_master_addrs=127.6.72.126:35623
--webserver_port=41179
--webserver_interface=127.6.72.67
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab
--principal=oryx/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:38181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.72.126:35623,127.6.72.125:38209
--tserver_master_addrs=127.6.72.126:35623,127.6.72.125:38209,127.6.72.124:36087 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:43:27.686648 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.669050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:39101 (local address 127.6.72.126:35623)
0925 00:43:27.669205 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.669209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.669975 (+ 766us) server_negotiation.cc:407] Connection header received
0925 00:43:27.671269 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.671272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.671333 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.671440 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.672742 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.673575 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.674705 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.674926 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.675043 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.681459 (+ 6416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.681463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.681467 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.681501 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.683657 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.684562 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.684568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.684570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.684680 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.685140 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.685146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.685149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.685398 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.685487 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.686295 (+ 808us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.686441 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 00:43:27.686674 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.668846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:40629 (local address 127.6.72.124:36087)
0925 00:43:27.669008 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.669013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.669972 (+ 959us) server_negotiation.cc:407] Connection header received
0925 00:43:27.671225 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.671228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.671284 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.671337 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.672781 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.673318 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.675157 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.675427 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.675530 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.681459 (+ 5929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.681463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.681466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.681503 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.683949 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.684658 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.684662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.684664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.684760 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.685154 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.685158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.685159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.685404 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.685490 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.686300 (+ 810us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.686465 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 00:43:27.686906 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.670529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:41925 (local address 127.6.72.125:38209)
0925 00:43:27.670659 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.670662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.670710 (+ 48us) server_negotiation.cc:407] Connection header received
0925 00:43:27.671225 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.671229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.671283 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.671337 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.672807 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.673581 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.675192 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.675557 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.675664 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.682895 (+ 7231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.682900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.682902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.682939 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.684797 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.685590 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.685594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.685595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.685688 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.686001 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.686005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.686006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.686263 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.686330 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.686658 (+ 328us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.686772 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 00:43:27.688488 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.669340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.66:39101)
0925 00:43:27.669817 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.669870 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.670988 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.671616 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.671624 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.672045 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.672595 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.672609 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.673738 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.673741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.674566 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.674574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.675092 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.675096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.675296 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.676050 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.676076 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.681193 (+ 5117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.683886 (+ 2693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.683895 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.683918 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.684428 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.684803 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.684806 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.684808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.685019 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.685557 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.685567 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.685996 (+ 429us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.686833 (+ 837us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.687390 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":199,"spinlock_wait_cycles":19200,"thread_start_us":126,"threads_started":1}
I20260925 00:43:27.688488 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.669340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:36087 (local address 127.6.72.66:40629)
0925 00:43:27.669817 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.669870 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.670990 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.671618 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.671625 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.672097 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.672632 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.672638 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.673461 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.673464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.675016 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.675026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.675712 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.675716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.675819 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.676302 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.676319 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.681193 (+ 4874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.684150 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.684156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.684161 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.684532 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.684876 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.684882 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.684887 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.685023 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.685559 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.685568 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.685996 (+ 428us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.686834 (+ 838us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.687296 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":233,"spinlock_wait_cycles":57856,"thread_start_us":139,"threads_started":1}
I20260925 00:43:27.689292 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.670382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:38209 (local address 127.6.72.66:41925)
0925 00:43:27.670636 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.670650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.670990 (+ 340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.671618 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.671625 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.672106 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.672629 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.672638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.673696 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.673699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.675099 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.675109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.675813 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.675817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.675962 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.676495 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.676509 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.682686 (+ 6177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.684910 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.684913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.684915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.685474 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.685804 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.685807 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.685808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.685889 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.686401 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.686407 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.686484 (+ 77us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.687671 (+ 1187us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.687812 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":150,"spinlock_wait_cycles":13440,"thread_start_us":99,"threads_started":1}
I20260925 00:43:27.689728 13637 heartbeater.cc:347] Connected to a master server at 127.6.72.126:35623
I20260925 00:43:27.689971 13637 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:27.690310 13638 heartbeater.cc:347] Connected to a master server at 127.6.72.125:38209
I20260925 00:43:27.690400 13638 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:27.690523 13637 heartbeater.cc:511] Master 127.6.72.126:35623 requested a full tablet report, sending...
I20260925 00:43:27.690605 13638 heartbeater.cc:511] Master 127.6.72.125:38209 requested a full tablet report, sending...
I20260925 00:43:27.691018 13636 heartbeater.cc:347] Connected to a master server at 127.6.72.124:36087
I20260925 00:43:27.691100 13636 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:27.691255 13636 heartbeater.cc:511] Master 127.6.72.124:36087 requested a full tablet report, sending...
I20260925 00:43:27.691589 12537 ts_manager.cc:194] Re-registered known tserver with Master: b569cd1d3ae043839c852ca947f15c50 (127.6.72.66:33187)
I20260925 00:43:27.691700 13137 ts_manager.cc:194] Registered new tserver with Master: b569cd1d3ae043839c852ca947f15c50 (127.6.72.66:33187)
I20260925 00:43:27.692615 13312 ts_manager.cc:194] Registered new tserver with Master: b569cd1d3ae043839c852ca947f15c50 (127.6.72.66:33187)
I20260925 00:43:27.693120 12537 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:39101
I20260925 00:43:27.716126 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.712689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40752 (local address 127.6.72.124:36087)
0925 00:43:27.712844 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.712848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.712864 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:27.713013 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.713016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.713076 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.713220 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:27.713491 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.714052 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.715045 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.715536 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.715642 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.715811 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.715949 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.716016 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 00:43:27.716157 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.712596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:36087 (local address 127.0.0.1:40752)
0925 00:43:27.712759 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.712777 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.712884 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.713237 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.713240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.713260 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:27.713367 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.713373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.714173 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.714180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.714915 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.714924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.715693 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.715698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.715805 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.715832 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.715926 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.716006 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 00:43:27.716909 13326 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260925 00:43:27.717334 13362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c635738475c448680441d391652e4c5. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.717479 13362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: This master's current role is: LEARNER
W20260925 00:43:27.805683 13643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:27.805972 13643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:27.806017 13643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:27.810209 13643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:27.810277 13643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:27.810338 13643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:27.817072 13643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab
--principal=oryx/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:46823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=41179
--tserver_master_addrs=127.6.72.126:35623,127.6.72.125:38209,127.6.72.124:36087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:27.818398 13643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:27.819696 13643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:27.830860 13649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:27.830860 13648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:27.830860 13651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:27.830914 13643 server_base.cc:1034] running on GCE node
I20260925 00:43:27.831562 13643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:27.832231 13643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:27.833472 13643 hybrid_clock.cc:648] HybridClock initialized: now 1790297007833453 us; error 35 us; skew 500 ppm
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:27.836540 13643 init.cc:377] Logged in from keytab as oryx/127.6.72.67@KRBTEST.COM (short username oryx)
I20260925 00:43:27.837595 13643 webserver.cc:460] Webserver started at http://127.6.72.67:41179/ using document root <none> and password file <none>
I20260925 00:43:27.838250 13643 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:27.838330 13643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:27.842821 13643 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:43:27.845134 13658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.846398 13643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:27.846544 13643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "fd2b33f21d8f40348ee6037d9642dbfd"
format_stamp: "Formatted at 2026-09-25 00:43:26 on dist-test-slave-w8nh"
server_key: "2d85a954273feee1703661d0805b802e"
server_key_iv: "385afa931cf36163795419f882b3fb75"
server_key_version: "encryptionkey@0"
I20260925 00:43:27.847118 13643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:27.870554 13643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:27.873653 13643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:27.873862 13643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:27.874470 13643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:27.875356 13643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:27.875430 13643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.875500 13643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:27.875552 13643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:27.885561 13643 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:46823
I20260925 00:43:27.885600 13771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:46823 every 8 connection(s)
I20260925 00:43:27.886660 13643 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:27.893589 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 13643
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.124@KRBTEST.COM
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.125@KRBTEST.COM
Sep 25 00:43:27 dist-test-slave-w8nh krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297007, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:27.903695 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.888722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:50707 (local address 127.6.72.124:36087)
0925 00:43:27.888835 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.888838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.890029 (+ 1191us) server_negotiation.cc:407] Connection header received
0925 00:43:27.891285 (+ 1256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.891289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.891353 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.891429 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.892994 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.893537 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.895168 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.895537 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.895669 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.899246 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.899253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.899259 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.899295 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.901084 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.901763 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.901767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.901769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.901860 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.902301 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.902307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.902310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.902536 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.902625 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.903385 (+ 760us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.903511 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 00:43:27.905846 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.890523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:57627 (local address 127.6.72.125:38209)
0925 00:43:27.890656 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.890660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.890799 (+ 139us) server_negotiation.cc:407] Connection header received
0925 00:43:27.891695 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.891699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.891745 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.891833 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.894623 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.895324 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.896674 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.896989 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.897097 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.900651 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.900655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.900657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.900711 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.903406 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.904283 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.904287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.904291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.904394 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.904740 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.904746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.904747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.904971 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.905070 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.905521 (+ 451us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.905664 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:43:27.905875 13777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.889257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:36087 (local address 127.6.72.67:50707)
0925 00:43:27.889850 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.889912 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.891043 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.891647 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.891654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.892300 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.892835 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.892848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.893694 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.893697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.895031 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.895040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.895715 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.895719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.895941 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.896903 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.896924 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.899017 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.901223 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.901229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.901250 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.901644 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.901985 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.901990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.901995 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.902192 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.902721 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.902729 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.903104 (+ 375us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.903899 (+ 795us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.904298 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":83,"spinlock_wait_cycles":31232,"thread_start_us":151,"threads_started":1}
I20260925 00:43:27.906694 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.890428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:38209 (local address 127.6.72.67:57627)
0925 00:43:27.890698 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.890713 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.891051 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.891871 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.891877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.892267 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.892847 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.892856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.895430 (+ 2574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.895434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.896356 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.896365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.897536 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.897543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.897681 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.898355 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.898374 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.900486 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.903519 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.903522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.903524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.903810 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.904511 (+ 701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.904516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.904520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.904612 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.905115 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.905118 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.905228 (+ 110us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.905333 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.906478 (+ 1145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":117,"spinlock_wait_cycles":8832,"thread_start_us":98,"threads_started":1}
I20260925 00:43:27.907289 13772 heartbeater.cc:347] Connected to a master server at 127.6.72.124:36087
I20260925 00:43:27.907358 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.889257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.67:41093)
0925 00:43:27.889850 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.889912 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.891043 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.891645 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.891653 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.892270 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:27.892848 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.892856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.895437 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.895440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.897140 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.897152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.897941 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.897945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.898053 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.898550 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:27.898565 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:27.900742 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:27.905239 (+ 4497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.905242 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.905244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.905504 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.905811 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:27.905814 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:27.905816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:27.905899 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:27.906814 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:27.906817 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:27.906909 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.907008 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.907177 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":958,"spinlock_wait_cycles":49536,"thread_start_us":138,"threads_started":1}
I20260925 00:43:27.907549 13772 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:27.907847 13774 heartbeater.cc:347] Connected to a master server at 127.6.72.125:38209
I20260925 00:43:27.907866 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.888626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:41093 (local address 127.6.72.126:35623)
0925 00:43:27.888775 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.888781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.890030 (+ 1249us) server_negotiation.cc:407] Connection header received
0925 00:43:27.891432 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.891435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.891496 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.891575 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.893710 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.894473 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.897280 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.897689 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.897794 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.900923 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.900929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.900932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.900962 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.905114 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.905616 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.905620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.905622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.905709 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.906016 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.906021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.906025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.906665 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.906768 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.907562 (+ 794us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.907697 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:43:27.907923 13774 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:27.908166 13772 heartbeater.cc:511] Master 127.6.72.124:36087 requested a full tablet report, sending...
I20260925 00:43:27.908514 13773 heartbeater.cc:347] Connected to a master server at 127.6.72.126:35623
I20260925 00:43:27.908586 13773 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:27.908732 13773 heartbeater.cc:511] Master 127.6.72.126:35623 requested a full tablet report, sending...
I20260925 00:43:27.908166 13774 heartbeater.cc:511] Master 127.6.72.125:38209 requested a full tablet report, sending...
I20260925 00:43:27.909271 13784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.896394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39346 (local address 127.6.72.126:35623)
0925 00:43:27.896790 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.896794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.896808 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:27.896844 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.896847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.896892 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.898611 (+ 1719us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.898748 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.899430 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.900984 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.901411 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.901526 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.903457 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.903463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.903469 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.903501 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.907084 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.907588 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.907591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.907593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.907682 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.907988 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.907991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.907992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.908208 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.908358 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.908974 (+ 616us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.909116 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":74,"threads_started":1}
I20260925 00:43:27.909485 12537 ts_manager.cc:194] Re-registered known tserver with Master: fd2b33f21d8f40348ee6037d9642dbfd (127.6.72.67:46823)
I20260925 00:43:27.909565 13137 ts_manager.cc:194] Registered new tserver with Master: fd2b33f21d8f40348ee6037d9642dbfd (127.6.72.67:46823)
I20260925 00:43:27.909577 13312 ts_manager.cc:194] Registered new tserver with Master: fd2b33f21d8f40348ee6037d9642dbfd (127.6.72.67:46823)
I20260925 00:43:27.910326 12537 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:41093
I20260925 00:43:27.912827 12537 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 00:43:27.915087 13785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.896696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58304 (local address 127.6.72.125:38209)
0925 00:43:27.900089 (+ 3393us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.900095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.902147 (+ 2052us) server_negotiation.cc:407] Connection header received
0925 00:43:27.902189 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.902192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.902249 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.902332 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.903285 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.904017 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.905167 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.905494 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.905602 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.911292 (+ 5690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.911297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.911299 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.911324 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.913263 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.913764 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.913767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.913769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.913853 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.914132 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.914135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.914137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.914334 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.914446 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.914780 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.914907 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2943,"thread_start_us":72,"threads_started":1}
W20260925 00:43:27.916581 12537 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:27.919387 13284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: NON_VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: true } } }
I20260925 00:43:27.919524 13284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:27.920612 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.898161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.6.72.124:36087)
0925 00:43:27.901707 (+ 3546us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.901713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.906390 (+ 4677us) server_negotiation.cc:407] Connection header received
0925 00:43:27.911453 (+ 5063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.911459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.911517 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.911608 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:27.912574 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.913413 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.914599 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.914916 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.915035 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.916716 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:27.916722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:27.916725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:27.916754 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:27.918363 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.918817 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.918821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.918823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.918933 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:27.919275 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:27.919279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:27.919280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:27.919532 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:27.919648 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.920317 (+ 669us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.920444 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3451,"thread_start_us":67,"threads_started":1}
I20260925 00:43:27.923097 13791 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5: attempting to promote NON_VOTER 3b718e270acb433babe5435e4ba571b4 to VOTER
I20260925 00:43:27.923457 13791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } }
I20260925 00:43:27.924650 13326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 LEARNER]: Refusing update from remote peer 4c635738475c448680441d391652e4c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260925 00:43:27.924649 13147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 FOLLOWER]: Refusing update from remote peer 4c635738475c448680441d391652e4c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260925 00:43:27.925057 13284 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 00:43:27.925263 13797 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260925 00:43:27.927457 13790 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3b718e270acb433babe5435e4ba571b4 (127.6.72.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:27.927938 13147 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3b718e270acb433babe5435e4ba571b4 (127.6.72.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:27.928025 13326 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3b718e270acb433babe5435e4ba571b4 (127.6.72.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:27.929126 13790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:27.929243 13790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:27.929368 13790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:27.929442 13790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:27.929488 13284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:27.929615 13284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:27.930418 13362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:27.930532 13362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:43:27.930768 13800 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c635738475c448680441d391652e4c5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:27.930922 13800 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:43:27.940291 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.934214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:33187 (local address 127.0.0.1:53830)
0925 00:43:27.934539 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.934555 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.934658 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.935854 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.935857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.935876 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:27.935989 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.935997 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.937384 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.937387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.938661 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.938673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.939836 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.939841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.939936 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.939961 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.940008 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.940098 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":48}
I20260925 00:43:27.941331 13801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.934821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:45781 (local address 127.0.0.1:53242)
0925 00:43:27.935131 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.935146 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.935237 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.936913 (+ 1676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.936916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.936933 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:27.937062 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.937068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.938673 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.938678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.939815 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.939823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.940926 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.940931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.941023 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.941047 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.941098 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.941167 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":22,"thread_start_us":159,"threads_started":1}
I20260925 00:43:27.941339 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.935744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59476 (local address 127.6.72.67:46823)
0925 00:43:27.936291 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.936349 (+ 58us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.936376 (+ 27us) server_negotiation.cc:407] Connection header received
0925 00:43:27.936458 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.936465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.936659 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.937093 (+ 434us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:27.937288 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.938109 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.939363 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.940325 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.940817 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.941001 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.941075 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.941177 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":142,"threads_started":1}
I20260925 00:43:27.941578 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.934822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.6.72.66:33187)
0925 00:43:27.935322 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.935334 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.935359 (+ 25us) server_negotiation.cc:407] Connection header received
0925 00:43:27.935448 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.935454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.935702 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.935890 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:27.936116 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.937268 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.938777 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.939704 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.941137 (+ 1433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.941283 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.941334 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.941432 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":124,"threads_started":1}
I20260925 00:43:27.941687 13804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.935317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:46823 (local address 127.0.0.1:59476)
0925 00:43:27.935576 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:27.935588 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:27.935690 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:27.936847 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:27.936850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:27.936872 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:27.937213 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.937217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.938268 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.938271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.939227 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:27.939236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.940811 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.940815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:27.940915 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.940941 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 00:43:27.941030 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 00:43:27.941111 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":230,"thread_start_us":109,"threads_started":1}
I20260925 00:43:27.942350 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:27.934965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53242 (local address 127.6.72.65:45781)
0925 00:43:27.936171 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:27.936185 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:27.936206 (+ 21us) server_negotiation.cc:407] Connection header received
0925 00:43:27.936303 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:27.936307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:27.936755 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:27.937335 (+ 580us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:27.937482 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.938580 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.939933 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:27.940794 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:27.941952 (+ 1158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:27.942085 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:27.942131 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 00:43:27.942220 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1083,"thread_start_us":56,"threads_started":1}
I20260925 00:43:27.944696 13433 tablet_service.cc:1467] Processing CreateTablet for tablet b0b227ca1f584104ae5949600c86f410 (DEFAULT_TABLE table=test-table [id=d23b5e83d80e492391860640490fb535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:27.944698 13570 tablet_service.cc:1467] Processing CreateTablet for tablet b0b227ca1f584104ae5949600c86f410 (DEFAULT_TABLE table=test-table [id=d23b5e83d80e492391860640490fb535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:27.944705 13706 tablet_service.cc:1467] Processing CreateTablet for tablet b0b227ca1f584104ae5949600c86f410 (DEFAULT_TABLE table=test-table [id=d23b5e83d80e492391860640490fb535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:27.945731 13433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b227ca1f584104ae5949600c86f410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:27.945779 13706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b227ca1f584104ae5949600c86f410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:27.945779 13570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0b227ca1f584104ae5949600c86f410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:27.953431 13806 tablet_bootstrap.cc:492] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: Bootstrap starting.
I20260925 00:43:27.954797 13807 tablet_bootstrap.cc:492] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: Bootstrap starting.
I20260925 00:43:27.956164 13806 tablet_bootstrap.cc:654] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:27.956918 13807 tablet_bootstrap.cc:654] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:27.956936 13806 log.cc:824] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:27.957626 13807 log.cc:824] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:27.958364 13808 tablet_bootstrap.cc:492] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: Bootstrap starting.
I20260925 00:43:27.958894 13806 tablet_bootstrap.cc:492] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: No bootstrap required, opened a new log
I20260925 00:43:27.959089 13806 ts_tablet_manager.cc:1397] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:43:27.959626 13807 tablet_bootstrap.cc:492] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: No bootstrap required, opened a new log
I20260925 00:43:27.959811 13807 ts_tablet_manager.cc:1397] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 00:43:27.960575 13808 tablet_bootstrap.cc:654] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:27.961418 13808 log.cc:824] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:27.963328 13806 raft_consensus.cc:348] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:27.963629 13806 raft_consensus.cc:374] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:27.963687 13806 raft_consensus.cc:729] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51d08a7b3a834c94982a8cb00ee067e9, State: Initialized, Role: FOLLOWER
I20260925 00:43:27.963722 13808 tablet_bootstrap.cc:492] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: No bootstrap required, opened a new log
I20260925 00:43:27.963937 13808 ts_tablet_manager.cc:1397] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:43:27.964155 13806 consensus_queue.cc:260] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:27.964325 13807 raft_consensus.cc:348] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:27.964582 13807 raft_consensus.cc:374] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:27.964640 13807 raft_consensus.cc:729] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2b33f21d8f40348ee6037d9642dbfd, State: Initialized, Role: FOLLOWER
I20260925 00:43:27.965052 13500 heartbeater.cc:503] Master 127.6.72.126:35623 was elected leader, sending a full tablet report...
I20260925 00:43:27.965096 13807 consensus_queue.cc:260] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:27.966372 13807 ts_tablet_manager.cc:1428] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 00:43:27.967118 13773 heartbeater.cc:503] Master 127.6.72.126:35623 was elected leader, sending a full tablet report...
I20260925 00:43:27.970216 13808 raft_consensus.cc:348] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:27.970554 13808 raft_consensus.cc:374] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:27.970625 13808 raft_consensus.cc:729] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b569cd1d3ae043839c852ca947f15c50, State: Initialized, Role: FOLLOWER
I20260925 00:43:27.970762 13806 ts_tablet_manager.cc:1428] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260925 00:43:27.971593 13808 consensus_queue.cc:260] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:27.972749 13808 ts_tablet_manager.cc:1428] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260925 00:43:27.974327 13637 heartbeater.cc:503] Master 127.6.72.126:35623 was elected leader, sending a full tablet report...
I20260925 00:43:28.079598 13814 raft_consensus.cc:482] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:28.079860 13814 raft_consensus.cc:504] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:28.081070 13814 leader_election.cc:288] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd2b33f21d8f40348ee6037d9642dbfd (127.6.72.67:46823), 51d08a7b3a834c94982a8cb00ee067e9 (127.6.72.65:45781)
I20260925 00:43:28.086436 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:28.081478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:51713 (local address 127.6.72.65:45781)
0925 00:43:28.081627 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:28.081633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:28.081862 (+ 229us) server_negotiation.cc:407] Connection header received
0925 00:43:28.081977 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:28.081984 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:28.082033 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:28.082105 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:28.082464 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.083297 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.085064 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.085692 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.085802 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:28.085866 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:28.086226 (+ 360us) server_negotiation.cc:299] Negotiation successful
0925 00:43:28.086301 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 00:43:28.086432 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:28.081603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:39197 (local address 127.6.72.67:46823)
0925 00:43:28.081759 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:28.081766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:28.081780 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:28.081857 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:28.081860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:28.081912 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:28.081993 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:28.082465 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.083297 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.085077 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.085697 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.085802 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:28.085866 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:28.086232 (+ 366us) server_negotiation.cc:299] Negotiation successful
0925 00:43:28.086301 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:43:28.086463 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:28.081387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:46823 (local address 127.6.72.66:39197)
0925 00:43:28.081529 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:28.081547 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:28.081676 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:28.082013 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:28.082018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:28.082110 (+ 92us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:28.082346 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:28.082351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.083418 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.083422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:28.084923 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:28.084935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.085833 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.085837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:28.086016 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:28.086131 (+ 115us) client_negotiation.cc:769] Sending connection context
0925 00:43:28.086199 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 00:43:28.086267 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":262}
I20260925 00:43:28.086472 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:28.081387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:45781 (local address 127.6.72.66:51713)
0925 00:43:28.081772 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:28.081792 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:28.081870 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:28.082143 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:28.082146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:28.082158 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:28.082343 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:28.082350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.083418 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.083422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:28.084923 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:28.084935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.085833 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.085837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:28.086019 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:28.086093 (+ 74us) client_negotiation.cc:769] Sending connection context
0925 00:43:28.086180 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 00:43:28.086268 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":425}
I20260925 00:43:28.087186 13726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b227ca1f584104ae5949600c86f410" candidate_uuid: "b569cd1d3ae043839c852ca947f15c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" is_pre_election: true
I20260925 00:43:28.087396 13453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b227ca1f584104ae5949600c86f410" candidate_uuid: "b569cd1d3ae043839c852ca947f15c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51d08a7b3a834c94982a8cb00ee067e9" is_pre_election: true
I20260925 00:43:28.087512 13726 raft_consensus.cc:2454] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b569cd1d3ae043839c852ca947f15c50 in term 0.
I20260925 00:43:28.087707 13453 raft_consensus.cc:2454] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b569cd1d3ae043839c852ca947f15c50 in term 0.
I20260925 00:43:28.088095 13523 leader_election.cc:302] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 51d08a7b3a834c94982a8cb00ee067e9, b569cd1d3ae043839c852ca947f15c50, fd2b33f21d8f40348ee6037d9642dbfd; no voters:
I20260925 00:43:28.088439 13814 raft_consensus.cc:2781] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:28.088518 13814 raft_consensus.cc:482] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:28.088549 13814 raft_consensus.cc:3037] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:28.089637 13814 raft_consensus.cc:504] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:28.090008 13814 leader_election.cc:288] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [CANDIDATE]: Term 1 election: Requested vote from peers fd2b33f21d8f40348ee6037d9642dbfd (127.6.72.67:46823), 51d08a7b3a834c94982a8cb00ee067e9 (127.6.72.65:45781)
I20260925 00:43:28.090389 13453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b227ca1f584104ae5949600c86f410" candidate_uuid: "b569cd1d3ae043839c852ca947f15c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51d08a7b3a834c94982a8cb00ee067e9"
I20260925 00:43:28.090469 13726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0b227ca1f584104ae5949600c86f410" candidate_uuid: "b569cd1d3ae043839c852ca947f15c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2b33f21d8f40348ee6037d9642dbfd"
I20260925 00:43:28.090538 13453 raft_consensus.cc:3037] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:28.090605 13726 raft_consensus.cc:3037] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:28.091565 13453 raft_consensus.cc:2454] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b569cd1d3ae043839c852ca947f15c50 in term 1.
I20260925 00:43:28.091578 13726 raft_consensus.cc:2454] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b569cd1d3ae043839c852ca947f15c50 in term 1.
I20260925 00:43:28.091850 13523 leader_election.cc:302] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b569cd1d3ae043839c852ca947f15c50, fd2b33f21d8f40348ee6037d9642dbfd; no voters:
I20260925 00:43:28.092056 13814 raft_consensus.cc:2781] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:28.092279 13814 raft_consensus.cc:686] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 1 LEADER]: Becoming Leader. State: Replica: b569cd1d3ae043839c852ca947f15c50, State: Running, Role: LEADER
I20260925 00:43:28.092617 13814 consensus_queue.cc:237] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:28.096307 12536 catalog_manager.cc:5495] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 reported cstate change: term changed from 0 to 1, leader changed from <none> to b569cd1d3ae043839c852ca947f15c50 (127.6.72.66). New cstate: current_term: 1 leader_uuid: "b569cd1d3ae043839c852ca947f15c50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } health_report { overall_health: UNKNOWN } } }
I20260925 00:43:28.134619 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:28.126756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.6.72.66:33187)
0925 00:43:28.126958 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:28.126966 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:28.126988 (+ 22us) server_negotiation.cc:407] Connection header received
0925 00:43:28.127102 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:28.127108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:28.127212 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:28.127362 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:28.128051 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.128925 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.131405 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.131834 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.132001 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:28.132910 (+ 909us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:28.133620 (+ 710us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:28.134188 (+ 568us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:28.134331 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 00:43:28.134442 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260925 00:43:28.138065 13775 tablet.cc:2369] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:43:28.143652 13453 raft_consensus.cc:1261] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9 [term 1 FOLLOWER]: Refusing update from remote peer b569cd1d3ae043839c852ca947f15c50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:28.143828 13726 raft_consensus.cc:1261] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd [term 1 FOLLOWER]: Refusing update from remote peer b569cd1d3ae043839c852ca947f15c50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:28.144645 13814 consensus_queue.cc:1035] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:28.144904 13815 consensus_queue.cc:1035] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:28.154863 13818 mvcc.cc:205] Tried to move back new op lower bound from 7333056545345998848 to 7333056545150025728. Current Snapshot: MvccSnapshot[applied={T|T < 7333056545345998848}]
I20260925 00:43:28.156435 13820 mvcc.cc:205] Tried to move back new op lower bound from 7333056545345998848 to 7333056545150025728. Current Snapshot: MvccSnapshot[applied={T|T < 7333056545345998848}]
I20260925 00:43:28.157883 13819 mvcc.cc:205] Tried to move back new op lower bound from 7333056545345998848 to 7333056545150025728. Current Snapshot: MvccSnapshot[applied={T|T < 7333056545345998848}]
I20260925 00:43:28.169304 12537 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:28.169503 12537 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:28.173750 12537 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4c635738475c448680441d391652e4c5: Sending DeleteTablet for 3 replicas of tablet b0b227ca1f584104ae5949600c86f410
I20260925 00:43:28.174716 13706 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b227ca1f584104ae5949600c86f410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:28 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:59476
I20260925 00:43:28.174857 13570 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b227ca1f584104ae5949600c86f410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:28 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:53830
I20260925 00:43:28.175915 13827 tablet_replica.cc:330] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: stopping tablet replica
I20260925 00:43:28.176306 13827 raft_consensus.cc:2229] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:28.176786 13827 raft_consensus.cc:2258] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:28.177109 13828 tablet_replica.cc:330] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: stopping tablet replica
I20260925 00:43:28.177232 13433 tablet_service.cc:1514] Processing DeleteTablet for tablet b0b227ca1f584104ae5949600c86f410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:28 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:53242
I20260925 00:43:28.177405 13828 raft_consensus.cc:2229] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:28.177520 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 12505
I20260925 00:43:28.178267 13830 tablet_replica.cc:330] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: stopping tablet replica
I20260925 00:43:28.178603 13830 raft_consensus.cc:2229] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:28.178996 13830 raft_consensus.cc:2258] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:28.179456 13827 ts_tablet_manager.cc:1905] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:28.180662 13830 ts_tablet_manager.cc:1905] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:28.182653 13828 raft_consensus.cc:2258] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:28.182315 13827 ts_tablet_manager.cc:1918] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:28.182987 13827 log.cc:1196] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/b0b227ca1f584104ae5949600c86f410
I20260925 00:43:28.183377 13827 ts_tablet_manager.cc:1939] T b0b227ca1f584104ae5949600c86f410 P b569cd1d3ae043839c852ca947f15c50: Deleting consensus metadata
I20260925 00:43:28.184418 13830 ts_tablet_manager.cc:1918] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:28.184574 13830 log.cc:1196] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/b0b227ca1f584104ae5949600c86f410
I20260925 00:43:28.184960 13830 ts_tablet_manager.cc:1939] T b0b227ca1f584104ae5949600c86f410 P 51d08a7b3a834c94982a8cb00ee067e9: Deleting consensus metadata
W20260925 00:43:28.185950 13525 connection.cc:535] server connection from 127.0.0.1:53830 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 00:43:28.186012 13389 connection.cc:441] server connection from 127.0.0.1:53242 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53242 (request call id 1) could send its response
I20260925 00:43:28.187817 13828 ts_tablet_manager.cc:1905] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:28.190932 13828 ts_tablet_manager.cc:1918] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:28.191043 13828 log.cc:1196] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/b0b227ca1f584104ae5949600c86f410
I20260925 00:43:28.191386 13828 ts_tablet_manager.cc:1939] T b0b227ca1f584104ae5949600c86f410 P fd2b33f21d8f40348ee6037d9642dbfd: Deleting consensus metadata
W20260925 00:43:28.192510 13661 connection.cc:441] server connection from 127.0.0.1:59476 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59476 (request call id 1) could send its response
I20260925 00:43:28.200714 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:28.188968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40768 (local address 127.6.72.124:36087)
0925 00:43:28.189097 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:28.189102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:28.189851 (+ 749us) server_negotiation.cc:407] Connection header received
0925 00:43:28.190050 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:28.190053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:28.190102 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:28.190173 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:28.191106 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.191889 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.193145 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.193456 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.193564 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:28.196146 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:28.196150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:28.196153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:28.196182 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:28.198426 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:28.199278 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:28.199283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:28.199284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:28.199367 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:28.199634 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:28.199637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:28.199638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:28.199838 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:28.199938 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:28.200430 (+ 492us) server_negotiation.cc:299] Negotiation successful
0925 00:43:28.200550 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 00:43:28.201331 13785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:28.176566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58318 (local address 127.6.72.125:38209)
0925 00:43:28.176676 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:28.176681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:28.188556 (+ 11875us) server_negotiation.cc:407] Connection header received
0925 00:43:28.189870 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:28.189874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:28.189927 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:28.189979 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:28.190921 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.191740 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.193330 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.193701 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.193862 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:28.196146 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:28.196159 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:28.196161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:28.196188 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:28.198346 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:28.199220 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:28.199224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:28.199228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:28.199318 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:28.199631 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:28.199634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:28.199636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:28.199838 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:28.199938 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:28.200430 (+ 492us) server_negotiation.cc:299] Negotiation successful
0925 00:43:28.201163 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":27}
I20260925 00:43:28.204197 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:28.189232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.6.72.125:38209)
0925 00:43:28.189342 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:28.189348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:28.193488 (+ 4140us) server_negotiation.cc:407] Connection header received
0925 00:43:28.193662 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:28.193666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:28.193712 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:28.193761 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:28.195934 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.196714 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.197870 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.198207 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.198275 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:28.199892 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:28.199896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:28.199898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:28.199922 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:28.201870 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:28.203016 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:28.203019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:28.203021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:28.203106 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:28.203389 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:28.203392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:28.203394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:28.203558 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:28.203701 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:28.203926 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 00:43:28.204034 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 00:43:28.210258 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:28.176657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.6.72.124:36087)
0925 00:43:28.176749 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:28.176752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:28.201180 (+ 24428us) server_negotiation.cc:407] Connection header received
0925 00:43:28.201955 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:28.201959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:28.202010 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:28.202058 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:28.202823 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.203621 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.204963 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:28.205317 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:28.205394 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:28.207001 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:28.207003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:28.207005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:28.207026 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:28.208607 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:28.209044 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:28.209050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:28.209055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:28.209194 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:28.209488 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:28.209490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:28.209491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:28.209649 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:28.209746 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:28.209989 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 00:43:28.210082 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 00:43:29.159140 13838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.158689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.65:46209)
0925 00:43:29.158966 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.159049 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":110,"threads_started":1}
W20260925 00:43:29.159498 13500 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:35623 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
I20260925 00:43:29.161616 13839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.161180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.66:35223)
0925 00:43:29.161421 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.161490 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":68,"threads_started":1}
W20260925 00:43:29.161924 13637 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:35623 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
I20260925 00:43:29.165189 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.164713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.67:37727)
0925 00:43:29.164986 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.165089 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":76,"threads_started":1}
W20260925 00:43:29.165577 13773 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:35623 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
I20260925 00:43:29.674350 13841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4c635738475c448680441d391652e4c5)
I20260925 00:43:29.674520 13841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } }
I20260925 00:43:29.675112 13841 leader_election.cc:288] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4c635738475c448680441d391652e4c5 (127.6.72.126:35623), a1af392529814d1cb61dca0eca097f5e (127.6.72.125:38209)
I20260925 00:43:29.675133 13842 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4c635738475c448680441d391652e4c5)
I20260925 00:43:29.675364 13842 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } }
I20260925 00:43:29.675961 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.675338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:57256)
0925 00:43:29.675713 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.675810 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":57,"threads_started":1}
W20260925 00:43:29.676230 13306 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
I20260925 00:43:29.676986 13847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.676640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:57272)
0925 00:43:29.676827 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.676896 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":52,"threads_started":1}
W20260925 00:43:29.677304 13123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
I20260925 00:43:29.677701 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.676994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:57286)
0925 00:43:29.677576 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.677626 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516}
W20260925 00:43:29.677922 13306 leader_election.cc:334] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4c635738475c448680441d391652e4c5 (127.6.72.126:35623): Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
I20260925 00:43:29.678363 13842 leader_election.cc:288] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4c635738475c448680441d391652e4c5 (127.6.72.126:35623), 3b718e270acb433babe5435e4ba571b4 (127.6.72.124:36087)
I20260925 00:43:29.678948 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.678675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:57308)
0925 00:43:29.678827 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.678885 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260925 00:43:29.679132 13123 leader_election.cc:334] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4c635738475c448680441d391652e4c5 (127.6.72.126:35623): Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
I20260925 00:43:29.680176 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.675745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:38209 (local address 127.0.0.1:48746)
0925 00:43:29.675997 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.676013 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:29.676132 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:29.676463 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:29.676465 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:29.676484 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:29.676616 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.676624 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.677779 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.677783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.679050 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.679060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.679764 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.679767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.679860 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.679887 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 00:43:29.679973 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 00:43:29.680052 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":55,"threads_started":1}
I20260925 00:43:29.680229 13845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.675826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.6.72.125:38209)
0925 00:43:29.676080 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:29.676084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:29.676099 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:43:29.676265 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:29.676268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:29.676328 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:29.676414 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:29.676716 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.677651 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.679199 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.679643 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.679765 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.679825 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:29.679989 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 00:43:29.680115 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":175,"thread_start_us":62,"threads_started":1}
I20260925 00:43:29.680788 13147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b718e270acb433babe5435e4ba571b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a1af392529814d1cb61dca0eca097f5e" is_pre_election: true
I20260925 00:43:29.680903 13147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b718e270acb433babe5435e4ba571b4 in term 1.
I20260925 00:43:29.681221 13306 leader_election.cc:302] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b718e270acb433babe5435e4ba571b4, a1af392529814d1cb61dca0eca097f5e; no voters: 4c635738475c448680441d391652e4c5
I20260925 00:43:29.681414 13841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 00:43:29.681476 13841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4c635738475c448680441d391652e4c5)
I20260925 00:43:29.681542 13841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 1 FOLLOWER]: Advancing to term 2
I20260925 00:43:29.682358 13847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.677053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:36087 (local address 127.0.0.1:45322)
0925 00:43:29.677236 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.677249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:29.677409 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:29.678933 (+ 1524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:29.678939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:29.678966 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:29.679109 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.679117 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.679927 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.679931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.681190 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.681199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.681958 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.681962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.682050 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.682072 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:43:29.682167 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 00:43:29.682235 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":38,"thread_start_us":68,"threads_started":1}
I20260925 00:43:29.682358 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.678197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.6.72.124:36087)
0925 00:43:29.678696 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:29.678706 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:29.678719 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:43:29.678761 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:29.678763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:29.678810 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:29.679224 (+ 414us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:29.679329 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.679805 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.681319 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.681840 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.681957 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.682011 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:29.682181 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 00:43:29.682248 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":104,"threads_started":1}
I20260925 00:43:29.682634 13841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } }
I20260925 00:43:29.682817 13326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1af392529814d1cb61dca0eca097f5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3b718e270acb433babe5435e4ba571b4" is_pre_election: true
I20260925 00:43:29.683020 13841 leader_election.cc:288] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [CANDIDATE]: Term 2 election: Requested vote from peers 4c635738475c448680441d391652e4c5 (127.6.72.126:35623), a1af392529814d1cb61dca0eca097f5e (127.6.72.125:38209)
I20260925 00:43:29.683075 13326 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a1af392529814d1cb61dca0eca097f5e in current term 2: Already voted for candidate 3b718e270acb433babe5435e4ba571b4 in this term.
I20260925 00:43:29.683372 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.683133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:57310)
0925 00:43:29.683254 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.683315 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 00:43:29.683472 13124 leader_election.cc:302] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1af392529814d1cb61dca0eca097f5e; no voters: 3b718e270acb433babe5435e4ba571b4, 4c635738475c448680441d391652e4c5
I20260925 00:43:29.683490 13147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3b718e270acb433babe5435e4ba571b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a1af392529814d1cb61dca0eca097f5e"
I20260925 00:43:29.683614 13147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 1 FOLLOWER]: Advancing to term 2
I20260925 00:43:29.684139 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.683916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:57316)
0925 00:43:29.684020 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.684072 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 00:43:29.684324 13306 leader_election.cc:334] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4c635738475c448680441d391652e4c5 (127.6.72.126:35623): Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
I20260925 00:43:29.684818 13147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b718e270acb433babe5435e4ba571b4 in term 2.
I20260925 00:43:29.684911 13842 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 00:43:29.685135 13306 leader_election.cc:302] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b718e270acb433babe5435e4ba571b4, a1af392529814d1cb61dca0eca097f5e; no voters: 4c635738475c448680441d391652e4c5
I20260925 00:43:29.685303 13841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 00:43:29.685554 13841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [term 2 LEADER]: Becoming Leader. State: Replica: 3b718e270acb433babe5435e4ba571b4, State: Running, Role: LEADER
I20260925 00:43:29.685817 13841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } }
I20260925 00:43:29.686867 13851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b718e270acb433babe5435e4ba571b4. Latest consensus state: current_term: 2 leader_uuid: "3b718e270acb433babe5435e4ba571b4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:29.686954 13851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:29.687254 13853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:29.689566 13853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:29.689900 13853 catalog_manager.cc:1259] Loaded cluster ID: 497d8ee0601248caa0af71534968b68a
I20260925 00:43:29.689944 13853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:29.690444 13853 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:29.690773 13853 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Loaded TSK: 0
I20260925 00:43:29.691397 13853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:43:29.739414 13312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:29.742712 13312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:29.745739 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.745483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:57326)
0925 00:43:29.745617 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.745667 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 00:43:29.745738 13147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [term 2 FOLLOWER]: Refusing update from remote peer 3b718e270acb433babe5435e4ba571b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260925 00:43:29.746295 13841 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260925 00:43:29.746562 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.746333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.0.0.1:57342)
0925 00:43:29.746447 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.746501 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 00:43:29.746676 13306 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 -> Peer 4c635738475c448680441d391652e4c5 (127.6.72.126:35623): Couldn't send request to peer 4c635738475c448680441d391652e4c5. Status: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 00:43:29.747874 13842 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b718e270acb433babe5435e4ba571b4. Latest consensus state: current_term: 2 leader_uuid: "3b718e270acb433babe5435e4ba571b4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:29.748020 13842 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:43:29.749225 13851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3b718e270acb433babe5435e4ba571b4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:29.749339 13851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:29.749555 13842 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3b718e270acb433babe5435e4ba571b4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4c635738475c448680441d391652e4c5" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35623 } } peers { permanent_uuid: "a1af392529814d1cb61dca0eca097f5e" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 38209 } attrs { promote: false } } peers { permanent_uuid: "3b718e270acb433babe5435e4ba571b4" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 36087 } attrs { promote: false } } }
I20260925 00:43:29.749691 13842 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af392529814d1cb61dca0eca097f5e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:43:29.759727 13862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.756034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60368 (local address 127.6.72.67:46823)
0925 00:43:29.756398 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:29.756403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:29.756417 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:29.756467 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:29.756470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:29.756517 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:29.756748 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:29.756961 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.757474 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.758770 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.759175 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.759298 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.759365 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:29.759516 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 00:43:29.759590 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":101,"threads_started":1}
I20260925 00:43:29.759725 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.755570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:46823 (local address 127.0.0.1:60368)
0925 00:43:29.755711 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.755726 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:29.755796 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:29.756658 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:29.756661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:29.756679 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:29.756867 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.756871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.757583 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.757586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.758642 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.758651 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.759293 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.759296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.759390 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.759412 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:43:29.759502 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 00:43:29.759575 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":129,"spinlock_wait_cycles":1408}
I20260925 00:43:29.760684 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.755358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:33187 (local address 127.0.0.1:43486)
0925 00:43:29.755680 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.755696 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:29.755800 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:29.756605 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:29.756610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:29.756633 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:29.756759 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.756766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.757906 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.757910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.759312 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.759321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.760181 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.760184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.760360 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.760396 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 00:43:29.760471 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 00:43:29.760550 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":147,"spinlock_wait_cycles":5632}
I20260925 00:43:29.760704 13861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.755844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:45781 (local address 127.0.0.1:56142)
0925 00:43:29.756091 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.756104 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:29.756184 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:29.756755 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:29.756758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:29.756773 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:29.756861 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.756866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.757892 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.757895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.759311 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.759321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.760207 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.760211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.760381 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.760407 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 00:43:29.760479 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 00:43:29.760554 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":140,"thread_start_us":125,"threads_started":1}
I20260925 00:43:29.760699 13860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.755470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43486 (local address 127.6.72.66:33187)
0925 00:43:29.756356 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:29.756360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:29.756371 (+ 11us) server_negotiation.cc:407] Connection header received
0925 00:43:29.756439 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:29.756442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:29.756490 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:29.756581 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:29.757021 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.757767 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.759452 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.760061 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.760183 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.760270 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:29.760488 (+ 218us) server_negotiation.cc:299] Negotiation successful
0925 00:43:29.760566 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":62,"threads_started":1}
I20260925 00:43:29.760707 13706 tablet_service.cc:1467] Processing CreateTablet for tablet d93c64f6065943d6bbb7b1baf1328d67 (DEFAULT_TABLE table=test-table [id=027956bcdc2b4ef5b2fa76c51a60b9e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:29.760987 13706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d93c64f6065943d6bbb7b1baf1328d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:29.761132 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.756282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56142 (local address 127.6.72.65:45781)
0925 00:43:29.756541 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:29.756547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:29.756563 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:29.756621 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:29.756624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:29.756671 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:29.756746 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:29.756978 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.757766 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.759452 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.760061 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.760327 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.760393 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:29.760961 (+ 568us) server_negotiation.cc:299] Negotiation successful
0925 00:43:29.761032 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":85,"threads_started":1}
I20260925 00:43:29.761646 13433 tablet_service.cc:1467] Processing CreateTablet for tablet d93c64f6065943d6bbb7b1baf1328d67 (DEFAULT_TABLE table=test-table [id=027956bcdc2b4ef5b2fa76c51a60b9e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:29.761897 13433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d93c64f6065943d6bbb7b1baf1328d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:29.763209 13864 tablet_bootstrap.cc:492] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd: Bootstrap starting.
I20260925 00:43:29.763996 13570 tablet_service.cc:1467] Processing CreateTablet for tablet d93c64f6065943d6bbb7b1baf1328d67 (DEFAULT_TABLE table=test-table [id=027956bcdc2b4ef5b2fa76c51a60b9e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:29.764309 13570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d93c64f6065943d6bbb7b1baf1328d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:29.764428 13865 tablet_bootstrap.cc:492] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9: Bootstrap starting.
I20260925 00:43:29.764509 13864 tablet_bootstrap.cc:654] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:29.765364 13865 tablet_bootstrap.cc:654] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:29.765888 13864 tablet_bootstrap.cc:492] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd: No bootstrap required, opened a new log
I20260925 00:43:29.766007 13864 ts_tablet_manager.cc:1397] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:29.766505 13865 tablet_bootstrap.cc:492] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9: No bootstrap required, opened a new log
I20260925 00:43:29.766462 13864 raft_consensus.cc:348] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:29.766605 13864 raft_consensus.cc:374] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:29.766638 13864 raft_consensus.cc:729] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2b33f21d8f40348ee6037d9642dbfd, State: Initialized, Role: FOLLOWER
I20260925 00:43:29.766671 13865 ts_tablet_manager.cc:1397] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 00:43:29.766811 13864 consensus_queue.cc:260] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:29.767144 13864 ts_tablet_manager.cc:1428] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:29.767141 13865 raft_consensus.cc:348] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:29.767261 13865 raft_consensus.cc:374] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:29.767292 13865 raft_consensus.cc:729] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51d08a7b3a834c94982a8cb00ee067e9, State: Initialized, Role: FOLLOWER
I20260925 00:43:29.767293 13868 tablet_bootstrap.cc:492] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50: Bootstrap starting.
I20260925 00:43:29.767410 13865 consensus_queue.cc:260] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:29.767771 13865 ts_tablet_manager.cc:1428] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 00:43:29.768947 13868 tablet_bootstrap.cc:654] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:29.770336 13772 heartbeater.cc:503] Master 127.6.72.124:36087 was elected leader, sending a full tablet report...
I20260925 00:43:29.770759 13871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.767603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.67:45405)
0925 00:43:29.770572 (+ 2969us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.770654 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2895,"thread_start_us":83,"threads_started":1}
I20260925 00:43:29.771369 13872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.768174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.65:36861)
0925 00:43:29.771198 (+ 3024us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.771257 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2951,"thread_start_us":111,"threads_started":1}
I20260925 00:43:29.771467 13871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.771185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.67:51591)
0925 00:43:29.771335 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.771393 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 00:43:29.772030 13499 heartbeater.cc:503] Master 127.6.72.124:36087 was elected leader, sending a full tablet report...
I20260925 00:43:29.772497 13868 tablet_bootstrap.cc:492] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50: No bootstrap required, opened a new log
I20260925 00:43:29.772625 13868 ts_tablet_manager.cc:1397] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 00:43:29.772781 13872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.772300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.65:40107)
0925 00:43:29.772625 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.772717 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
I20260925 00:43:29.773111 13868 raft_consensus.cc:348] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:29.773370 13868 raft_consensus.cc:374] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:29.773433 13868 raft_consensus.cc:729] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b569cd1d3ae043839c852ca947f15c50, State: Initialized, Role: FOLLOWER
I20260925 00:43:29.773604 13868 consensus_queue.cc:260] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:29.774019 13868 ts_tablet_manager.cc:1428] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260925 00:43:29.774937 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.774493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.66:47827)
0925 00:43:29.774770 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.774843 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":116,"threads_started":1}
I20260925 00:43:29.775339 13636 heartbeater.cc:503] Master 127.6.72.124:36087 was elected leader, sending a full tablet report...
I20260925 00:43:29.775663 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.775386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.66:56081)
0925 00:43:29.775538 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.775601 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260925 00:43:29.834059 13869 raft_consensus.cc:482] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:29.834223 13869 raft_consensus.cc:504] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:29.835638 13869 leader_election.cc:288] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b569cd1d3ae043839c852ca947f15c50 (127.6.72.66:33187), 51d08a7b3a834c94982a8cb00ee067e9 (127.6.72.65:45781)
I20260925 00:43:29.839557 13871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.835825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:33187 (local address 127.6.72.67:45763)
0925 00:43:29.836011 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.836027 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:29.836175 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:29.836671 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:29.836673 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:29.836689 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:29.836787 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.836791 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.837557 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.837563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.838440 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.838449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.839122 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.839126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.839231 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.839250 (+ 19us) client_negotiation.cc:769] Sending connection context
0925 00:43:29.839342 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 00:43:29.839415 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260925 00:43:29.839557 13860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.835977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:45763 (local address 127.6.72.66:33187)
0925 00:43:29.836136 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:29.836142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:29.836161 (+ 19us) server_negotiation.cc:407] Connection header received
0925 00:43:29.836357 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:29.836363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:29.836424 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:29.836477 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:29.836905 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.837440 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.838572 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.838982 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.839107 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.839172 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:29.839358 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 00:43:29.839433 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 00:43:29.839864 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.836293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:56747 (local address 127.6.72.65:45781)
0925 00:43:29.836465 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:29.836470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:29.836485 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:43:29.836696 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:29.836706 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:29.836756 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:29.836852 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:29.837115 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.837800 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.838805 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.839245 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.839376 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.839443 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:29.839623 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 00:43:29.839723 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 00:43:29.840123 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.835869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:45781 (local address 127.6.72.67:56747)
0925 00:43:29.836390 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.836412 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:29.836502 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:29.836861 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:29.836864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:29.836882 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:29.836990 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.836996 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.837912 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.837917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.838670 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:29.838679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.839376 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.839380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:29.839478 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.839505 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 00:43:29.839608 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 00:43:29.839847 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":130,"threads_started":1}
I20260925 00:43:29.840091 13590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d93c64f6065943d6bbb7b1baf1328d67" candidate_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b569cd1d3ae043839c852ca947f15c50" is_pre_election: true
I20260925 00:43:29.840384 13590 raft_consensus.cc:2454] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd2b33f21d8f40348ee6037d9642dbfd in term 0.
I20260925 00:43:29.840662 13453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d93c64f6065943d6bbb7b1baf1328d67" candidate_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51d08a7b3a834c94982a8cb00ee067e9" is_pre_election: true
I20260925 00:43:29.840832 13453 raft_consensus.cc:2454] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd2b33f21d8f40348ee6037d9642dbfd in term 0.
I20260925 00:43:29.840924 13659 leader_election.cc:302] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b569cd1d3ae043839c852ca947f15c50, fd2b33f21d8f40348ee6037d9642dbfd; no voters:
I20260925 00:43:29.841213 13869 raft_consensus.cc:2781] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:29.841286 13869 raft_consensus.cc:482] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:29.841321 13869 raft_consensus.cc:3037] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:29.842635 13869 raft_consensus.cc:504] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:29.843089 13869 leader_election.cc:288] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [CANDIDATE]: Term 1 election: Requested vote from peers b569cd1d3ae043839c852ca947f15c50 (127.6.72.66:33187), 51d08a7b3a834c94982a8cb00ee067e9 (127.6.72.65:45781)
I20260925 00:43:29.843433 13590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d93c64f6065943d6bbb7b1baf1328d67" candidate_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b569cd1d3ae043839c852ca947f15c50"
I20260925 00:43:29.843582 13453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d93c64f6065943d6bbb7b1baf1328d67" candidate_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51d08a7b3a834c94982a8cb00ee067e9"
I20260925 00:43:29.843724 13453 raft_consensus.cc:3037] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:29.843693 13590 raft_consensus.cc:3037] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:29.844790 13453 raft_consensus.cc:2454] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd2b33f21d8f40348ee6037d9642dbfd in term 1.
I20260925 00:43:29.844825 13590 raft_consensus.cc:2454] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd2b33f21d8f40348ee6037d9642dbfd in term 1.
I20260925 00:43:29.845144 13659 leader_election.cc:302] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b569cd1d3ae043839c852ca947f15c50, fd2b33f21d8f40348ee6037d9642dbfd; no voters:
I20260925 00:43:29.845355 13869 raft_consensus.cc:2781] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:29.845566 13869 raft_consensus.cc:686] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 1 LEADER]: Becoming Leader. State: Replica: fd2b33f21d8f40348ee6037d9642dbfd, State: Running, Role: LEADER
I20260925 00:43:29.845852 13869 consensus_queue.cc:237] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } }
I20260925 00:43:29.847357 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.847135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.67:46271)
0925 00:43:29.847236 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.847293 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 00:43:29.848631 13313 catalog_manager.cc:5495] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd reported cstate change: term changed from 0 to 1, leader changed from <none> to fd2b33f21d8f40348ee6037d9642dbfd (127.6.72.67). New cstate: current_term: 1 leader_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd2b33f21d8f40348ee6037d9642dbfd" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 } health_report { overall_health: UNKNOWN } } }
I20260925 00:43:29.863677 13862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.860193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60374 (local address 127.6.72.67:46823)
0925 00:43:29.860380 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:29.860383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:29.860397 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:29.860468 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:29.860470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:29.860540 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:29.860630 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:29.860890 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.861397 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.862279 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:29.862496 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:29.862590 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:29.862814 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:29.863264 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:29.863352 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:29.863439 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 00:43:29.863488 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260925 00:43:29.866590 13453 raft_consensus.cc:1261] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9 [term 1 FOLLOWER]: Refusing update from remote peer fd2b33f21d8f40348ee6037d9642dbfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:29.866865 13590 raft_consensus.cc:1261] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50 [term 1 FOLLOWER]: Refusing update from remote peer fd2b33f21d8f40348ee6037d9642dbfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:29.867183 13877 consensus_queue.cc:1035] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "51d08a7b3a834c94982a8cb00ee067e9" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:29.867436 13869 consensus_queue.cc:1035] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b569cd1d3ae043839c852ca947f15c50" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:29.869771 13872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.869313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.65:58853)
0925 00:43:29.869607 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.869670 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221}
I20260925 00:43:29.870606 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.870236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.66:49985)
0925 00:43:29.870478 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.870533 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260925 00:43:29.872823 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.872490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.67:43999)
0925 00:43:29.872700 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.872753 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260925 00:43:29.874013 13872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.873745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.65:58803)
0925 00:43:29.873893 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.873942 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260925 00:43:29.876466 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.876170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.66:56541)
0925 00:43:29.876341 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.876394 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260925 00:43:29.876868 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:29.876593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35623 (local address 127.6.72.67:55389)
0925 00:43:29.876734 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:29.876802 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 00:43:29.882114 13313 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:29.882293 13313 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:29.890865 13313 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b718e270acb433babe5435e4ba571b4: Sending DeleteTablet for 3 replicas of tablet d93c64f6065943d6bbb7b1baf1328d67
I20260925 00:43:29.891736 13706 tablet_service.cc:1514] Processing DeleteTablet for tablet d93c64f6065943d6bbb7b1baf1328d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:29 UTC) from {username='oryx', principal='oryx/127.6.72.124@KRBTEST.COM'} at 127.0.0.1:60368
I20260925 00:43:29.891831 13570 tablet_service.cc:1514] Processing DeleteTablet for tablet d93c64f6065943d6bbb7b1baf1328d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:29 UTC) from {username='oryx', principal='oryx/127.6.72.124@KRBTEST.COM'} at 127.0.0.1:43486
I20260925 00:43:29.891912 13433 tablet_service.cc:1514] Processing DeleteTablet for tablet d93c64f6065943d6bbb7b1baf1328d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:29 UTC) from {username='oryx', principal='oryx/127.6.72.124@KRBTEST.COM'} at 127.0.0.1:56142
I20260925 00:43:29.892385 13889 tablet_replica.cc:330] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd: stopping tablet replica
I20260925 00:43:29.892504 13891 tablet_replica.cc:330] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9: stopping tablet replica
I20260925 00:43:29.892639 13891 raft_consensus.cc:2229] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:29.892657 13889 raft_consensus.cc:2229] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:29.892776 13891 raft_consensus.cc:2258] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:29.893321 13889 raft_consensus.cc:2258] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:29.893752 13891 ts_tablet_manager.cc:1905] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:29.894032 13889 ts_tablet_manager.cc:1905] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:29.896147 13891 ts_tablet_manager.cc:1918] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:29.896225 13891 log.cc:1196] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/d93c64f6065943d6bbb7b1baf1328d67
I20260925 00:43:29.896529 13891 ts_tablet_manager.cc:1939] T d93c64f6065943d6bbb7b1baf1328d67 P 51d08a7b3a834c94982a8cb00ee067e9: Deleting consensus metadata
I20260925 00:43:29.896812 13889 ts_tablet_manager.cc:1918] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:29.896943 13889 log.cc:1196] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/d93c64f6065943d6bbb7b1baf1328d67
I20260925 00:43:29.897219 13307 catalog_manager.cc:4841] TS 51d08a7b3a834c94982a8cb00ee067e9 (127.6.72.65:45781): tablet d93c64f6065943d6bbb7b1baf1328d67 (table test-table [id=027956bcdc2b4ef5b2fa76c51a60b9e4]) successfully deleted
I20260925 00:43:29.897279 13889 ts_tablet_manager.cc:1939] T d93c64f6065943d6bbb7b1baf1328d67 P fd2b33f21d8f40348ee6037d9642dbfd: Deleting consensus metadata
I20260925 00:43:29.897994 13304 catalog_manager.cc:4841] TS fd2b33f21d8f40348ee6037d9642dbfd (127.6.72.67:46823): tablet d93c64f6065943d6bbb7b1baf1328d67 (table test-table [id=027956bcdc2b4ef5b2fa76c51a60b9e4]) successfully deleted
I20260925 00:43:29.900022 13890 tablet_replica.cc:330] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50: stopping tablet replica
I20260925 00:43:29.900208 13890 raft_consensus.cc:2229] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:29.900321 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 13281
I20260925 00:43:29.900398 13890 raft_consensus.cc:2258] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:29.903337 13890 ts_tablet_manager.cc:1905] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:29.906396 13890 ts_tablet_manager.cc:1918] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:29.906489 13890 log.cc:1196] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/d93c64f6065943d6bbb7b1baf1328d67
I20260925 00:43:29.906773 13890 ts_tablet_manager.cc:1939] T d93c64f6065943d6bbb7b1baf1328d67 P b569cd1d3ae043839c852ca947f15c50: Deleting consensus metadata
I20260925 00:43:29.907677 13304 catalog_manager.cc:4841] TS b569cd1d3ae043839c852ca947f15c50 (127.6.72.66:33187): tablet d93c64f6065943d6bbb7b1baf1328d67 (table test-table [id=027956bcdc2b4ef5b2fa76c51a60b9e4]) successfully deleted
I20260925 00:43:29.907938 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 13507
I20260925 00:43:29.915774 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 13643
I20260925 00:43:29.922021 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 13001
I20260925 00:43:29.927903 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 13109
2026-09-25T00:43:29Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5480 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:29 dist-test-slave-w8nh krb5kdc[13894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:29 dist-test-slave-w8nh krb5kdc[13894](info): set up 2 sockets
Sep 25 00:43:29 dist-test-slave-w8nh krb5kdc[13894](info): commencing operation
krb5kdc: starting...
W20260925 00:43:31.021015 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:31 dist-test-slave-w8nh krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.126.keytab.
Entry for principal oryx/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.126.keytab.
I20260925 00:43:31.156665 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:45817
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.126.keytab
--principal=oryx/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:37449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:45817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:31.269114 13907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:31.269399 13907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:31.269439 13907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:31.273865 13907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:31.273927 13907 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 00:43:31.273944 13907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:31.273963 13907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:31.273979 13907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:31.273995 13907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:31.279486 13907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:37449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:45817
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.126.keytab
--principal=oryx/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:45817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:31.280664 13907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:31.281690 13907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:31.290835 13912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:31.290802 13915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:31.290875 13907 server_base.cc:1034] running on GCE node
W20260925 00:43:31.290835 13913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:31.291635 13907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:31.292958 13907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:31.294148 13907 hybrid_clock.cc:648] HybridClock initialized: now 1790297011294129 us; error 27 us; skew 500 ppm
Sep 25 00:43:31 dist-test-slave-w8nh krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297011, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:31.297372 13907 init.cc:377] Logged in from keytab as oryx/127.6.72.126@KRBTEST.COM (short username oryx)
I20260925 00:43:31.298242 13907 webserver.cc:460] Webserver started at http://127.6.72.126:41183/ using document root <none> and password file <none>
I20260925 00:43:31.298768 13907 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:31.298826 13907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:31.299062 13907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:31.301195 13907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "d84b0aafaaec4bd792811e7cc3821453"
format_stamp: "Formatted at 2026-09-25 00:43:31 on dist-test-slave-w8nh"
server_key: "52b09a35897d12ee07646e37dbdae112"
server_key_iv: "58fda2b041033937e962299c7b45b237"
server_key_version: "encryptionkey@0"
I20260925 00:43:31.301837 13907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "d84b0aafaaec4bd792811e7cc3821453"
format_stamp: "Formatted at 2026-09-25 00:43:31 on dist-test-slave-w8nh"
server_key: "52b09a35897d12ee07646e37dbdae112"
server_key_iv: "58fda2b041033937e962299c7b45b237"
server_key_version: "encryptionkey@0"
I20260925 00:43:31.305760 13907 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:43:31.308475 13922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:31.309837 13907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 00:43:31.309967 13907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "d84b0aafaaec4bd792811e7cc3821453"
format_stamp: "Formatted at 2026-09-25 00:43:31 on dist-test-slave-w8nh"
server_key: "52b09a35897d12ee07646e37dbdae112"
server_key_iv: "58fda2b041033937e962299c7b45b237"
server_key_version: "encryptionkey@0"
I20260925 00:43:31.310076 13907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:31.326552 13907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:31.329862 13907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:31.330067 13907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:31.338593 13907 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:45817
I20260925 00:43:31.338617 13984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:45817 every 8 connection(s)
I20260925 00:43:31.339581 13907 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:31.342502 13985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:31.343482 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 13907
I20260925 00:43:31.343582 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:31.343888 6433 external_mini_cluster.cc:1428] Setting key 789ab01fa35738c42d4e441df1f0cb38
I20260925 00:43:31.349036 13985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453: Bootstrap starting.
I20260925 00:43:31.351765 13985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:31.352774 13985 log.cc:824] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453: Log is configured to *not* fsync() on all Append() calls
Sep 25 00:43:31 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:31.354794 13985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453: No bootstrap required, opened a new log
I20260925 00:43:31.357836 13985 raft_consensus.cc:348] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84b0aafaaec4bd792811e7cc3821453" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45817 } }
I20260925 00:43:31.358094 13985 raft_consensus.cc:374] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:31.358168 13985 raft_consensus.cc:729] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84b0aafaaec4bd792811e7cc3821453, State: Initialized, Role: FOLLOWER
I20260925 00:43:31.358626 13985 consensus_queue.cc:260] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84b0aafaaec4bd792811e7cc3821453" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45817 } }
I20260925 00:43:31.358747 13985 raft_consensus.cc:388] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:31.358830 13985 raft_consensus.cc:482] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:31.358929 13985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:31.359468 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:31.345279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60604 (local address 127.6.72.126:45817)
0925 00:43:31.345700 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:31.345728 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:31.345763 (+ 35us) server_negotiation.cc:407] Connection header received
0925 00:43:31.346379 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:31.346395 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:31.346727 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:31.346999 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:31.347888 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.348932 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.350035 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.350496 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.350614 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:31.353827 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:31.353832 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:31.353846 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:31.353874 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:31.356211 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:31.356732 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:31.356740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:31.356750 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:31.356861 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:31.357164 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:31.357167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:31.357169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:31.357569 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:31.357799 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:31.358081 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 00:43:31.358550 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":150,"threads_started":1}
I20260925 00:43:31.359941 13985 raft_consensus.cc:504] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84b0aafaaec4bd792811e7cc3821453" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45817 } }
I20260925 00:43:31.360330 13985 leader_election.cc:302] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d84b0aafaaec4bd792811e7cc3821453; no voters:
I20260925 00:43:31.360724 13985 leader_election.cc:288] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:31.360839 13990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:31.361049 13990 raft_consensus.cc:686] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [term 1 LEADER]: Becoming Leader. State: Replica: d84b0aafaaec4bd792811e7cc3821453, State: Running, Role: LEADER
I20260925 00:43:31.361388 13990 consensus_queue.cc:237] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84b0aafaaec4bd792811e7cc3821453" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45817 } }
I20260925 00:43:31.362023 13985 sys_catalog.cc:564] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:31.362622 13992 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d84b0aafaaec4bd792811e7cc3821453. Latest consensus state: current_term: 1 leader_uuid: "d84b0aafaaec4bd792811e7cc3821453" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84b0aafaaec4bd792811e7cc3821453" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45817 } } }
I20260925 00:43:31.362838 13992 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:31.363129 13991 sys_catalog.cc:455] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d84b0aafaaec4bd792811e7cc3821453" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d84b0aafaaec4bd792811e7cc3821453" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45817 } } }
I20260925 00:43:31.363289 13991 sys_catalog.cc:458] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:31.363296 13999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:31.366650 13999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:31.373191 13999 catalog_manager.cc:1347] Generated new cluster ID: 4747ff541b664b818f0532c7d2ac9a0a
I20260925 00:43:31.373265 13999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:31.397039 13999 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:31.398355 13999 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:31.409483 13999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453: Generated new TSK 0
I20260925 00:43:31.410233 13999 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab.
Entry for principal oryx/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab.
I20260925 00:43:31.451231 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:45817
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab
--principal=oryx/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:37449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:31.583400 14011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:31.583671 14011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:31.583717 14011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:31.587783 14011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:31.587846 14011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:31.587888 14011 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 00:43:31.587921 14011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:31.593166 14011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:37449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab
--principal=oryx/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.72.126:45817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:31.594198 14011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:31.595171 14011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:31.604950 14016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:31.604950 14017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:31.605002 14019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:31.605336 14011 server_base.cc:1034] running on GCE node
I20260925 00:43:31.605751 14011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:31.606381 14011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:31.607571 14011 hybrid_clock.cc:648] HybridClock initialized: now 1790297011607556 us; error 36 us; skew 500 ppm
Sep 25 00:43:31 dist-test-slave-w8nh krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297011, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:31.610289 14011 init.cc:377] Logged in from keytab as oryx/127.6.72.65@KRBTEST.COM (short username oryx)
I20260925 00:43:31.611152 14011 webserver.cc:460] Webserver started at http://127.6.72.65:45397/ using document root <none> and password file <none>
I20260925 00:43:31.611727 14011 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:31.611817 14011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:31.612048 14011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:31.614470 14011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "8e495c846981446c8c6b099b0a1dd2c6"
format_stamp: "Formatted at 2026-09-25 00:43:31 on dist-test-slave-w8nh"
server_key: "349fabd466c774a470e1348665e124a5"
server_key_iv: "0425505d92312d44fc96d21ea5bcea2e"
server_key_version: "encryptionkey@0"
I20260925 00:43:31.615120 14011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "8e495c846981446c8c6b099b0a1dd2c6"
format_stamp: "Formatted at 2026-09-25 00:43:31 on dist-test-slave-w8nh"
server_key: "349fabd466c774a470e1348665e124a5"
server_key_iv: "0425505d92312d44fc96d21ea5bcea2e"
server_key_version: "encryptionkey@0"
I20260925 00:43:31.619020 14011 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260925 00:43:31.621627 14026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:31.622841 14011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:31.622980 14011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "8e495c846981446c8c6b099b0a1dd2c6"
format_stamp: "Formatted at 2026-09-25 00:43:31 on dist-test-slave-w8nh"
server_key: "349fabd466c774a470e1348665e124a5"
server_key_iv: "0425505d92312d44fc96d21ea5bcea2e"
server_key_version: "encryptionkey@0"
I20260925 00:43:31.623121 14011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:31.647482 14011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:31.650643 14011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:31.650863 14011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:31.652452 14011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:31.652527 14011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:31.652612 14011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:31.652658 14011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 00:43:31 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297011, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:31.664106 14011 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:35161
I20260925 00:43:31.664229 14141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:35161 every 8 connection(s)
I20260925 00:43:31.665093 14011 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:31.667793 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:31.653224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:42893 (local address 127.6.72.126:45817)
0925 00:43:31.653387 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:31.653392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:31.654338 (+ 946us) server_negotiation.cc:407] Connection header received
0925 00:43:31.655623 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:31.655630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:31.655698 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:31.655765 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:31.657761 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.658417 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.660369 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.660702 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.660810 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:31.664022 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:31.664028 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:31.664031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:31.664061 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:31.665764 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:31.666359 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:31.666361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:31.666363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:31.666442 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:31.666761 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:31.666764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:31.666765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:31.666983 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:31.667099 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:31.667535 (+ 436us) server_negotiation.cc:299] Negotiation successful
0925 00:43:31.667669 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 00:43:31.668633 14035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:31.653636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:45817 (local address 127.6.72.65:42893)
0925 00:43:31.654144 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:31.654205 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:31.655344 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:31.656235 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:31.656273 (+ 38us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:31.656778 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:31.657558 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:31.657582 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.658582 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.658588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:31.660211 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:31.660225 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.660881 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.660886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:31.661113 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:31.661911 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:31.661934 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:31.663836 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:31.665894 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:31.665900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:31.665912 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:31.666259 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:31.666527 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:31.666529 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:31.666530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:31.666648 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:31.667088 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:31.667093 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:31.667322 (+ 229us) client_negotiation.cc:769] Sending connection context
0925 00:43:31.667476 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 00:43:31.667736 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":122,"threads_started":1}
I20260925 00:43:31.669014 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 14011
I20260925 00:43:31.669160 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:31.669528 6433 external_mini_cluster.cc:1428] Setting key 1eb581fe4ced5e8e5acb1eac4fcb0e8f
I20260925 00:43:31.670141 14142 heartbeater.cc:347] Connected to a master server at 127.6.72.126:45817
I20260925 00:43:31.670291 14142 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:31.670840 14142 heartbeater.cc:511] Master 127.6.72.126:45817 requested a full tablet report, sending...
I20260925 00:43:31.672350 13939 ts_manager.cc:194] Registered new tserver with Master: 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161)
I20260925 00:43:31.673703 13939 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:42893
WARNING: no policy specified for oryx/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:43:31.681602 14149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:31.672609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:45817 (local address 127.6.72.65:57153)
0925 00:43:31.674386 (+ 1777us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:31.674398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:31.674506 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:31.674781 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:31.674784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:31.675021 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:31.675146 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:31.675154 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.676165 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.676168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:31.676795 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:31.676803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.677412 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.677414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:31.677483 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:31.678086 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:31.678097 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:31.678446 (+ 349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:31.680209 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:31.680212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:31.680214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:31.680463 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:31.680789 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:31.680791 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:31.680792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:31.680865 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:31.681187 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:31.681188 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:31.681261 (+ 73us) client_negotiation.cc:769] Sending connection context
0925 00:43:31.681339 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 00:43:31.681447 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1682,"thread_start_us":111,"threads_started":1}
I20260925 00:43:31.681787 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:31.672666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:57153 (local address 127.6.72.126:45817)
0925 00:43:31.672770 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:31.672774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:31.674464 (+ 1690us) server_negotiation.cc:407] Connection header received
0925 00:43:31.674598 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:31.674604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:31.674658 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:31.674752 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:31.675283 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.675995 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.676913 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.677292 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.677405 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:31.678595 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:31.678600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:31.678607 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:31.678633 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:31.680111 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:31.680581 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:31.680587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:31.680592 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:31.680696 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:31.680941 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:31.680943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:31.680944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:31.681105 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:31.681185 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:31.681484 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 00:43:31.681584 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab.
Entry for principal oryx/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab.
I20260925 00:43:31.700182 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:45817
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab
--principal=oryx/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:37449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:31.813573 14151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:31.813916 14151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:31.814013 14151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:31.818238 14151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:31.818325 14151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:31.818393 14151 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 00:43:31.818480 14151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:31.823827 14151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:37449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab
--principal=oryx/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.72.126:45817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:31.824954 14151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:31.826064 14151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:31.835469 14156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:31.835453 14159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:31.835456 14157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:31.835892 14151 server_base.cc:1034] running on GCE node
I20260925 00:43:31.836349 14151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:31.836972 14151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:31.838178 14151 hybrid_clock.cc:648] HybridClock initialized: now 1790297011838144 us; error 49 us; skew 500 ppm
Sep 25 00:43:31 dist-test-slave-w8nh krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297011, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:31.841003 14151 init.cc:377] Logged in from keytab as oryx/127.6.72.66@KRBTEST.COM (short username oryx)
I20260925 00:43:31.841871 14151 webserver.cc:460] Webserver started at http://127.6.72.66:37417/ using document root <none> and password file <none>
I20260925 00:43:31.842439 14151 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:31.842530 14151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:31.842762 14151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:31.844817 14151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "4fcad5c64a614a4c9d379852df217d27"
format_stamp: "Formatted at 2026-09-25 00:43:31 on dist-test-slave-w8nh"
server_key: "1ec2626be9568293d13cfdfa4188c74c"
server_key_iv: "203dde2b02dd119946ddf5763b8cc1d2"
server_key_version: "encryptionkey@0"
I20260925 00:43:31.845371 14151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "4fcad5c64a614a4c9d379852df217d27"
format_stamp: "Formatted at 2026-09-25 00:43:31 on dist-test-slave-w8nh"
server_key: "1ec2626be9568293d13cfdfa4188c74c"
server_key_iv: "203dde2b02dd119946ddf5763b8cc1d2"
server_key_version: "encryptionkey@0"
I20260925 00:43:31.849061 14151 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 00:43:31.851468 14166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:31.852792 14151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:31.852926 14151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "4fcad5c64a614a4c9d379852df217d27"
format_stamp: "Formatted at 2026-09-25 00:43:31 on dist-test-slave-w8nh"
server_key: "1ec2626be9568293d13cfdfa4188c74c"
server_key_iv: "203dde2b02dd119946ddf5763b8cc1d2"
server_key_version: "encryptionkey@0"
I20260925 00:43:31.853084 14151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:31.873400 14151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:31.876945 14151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:31.877166 14151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:31.878683 14151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:31.878755 14151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:31.878832 14151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:31.878883 14151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 00:43:31 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297011, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:31.890002 14151 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:45523
I20260925 00:43:31.890064 14281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:45523 every 8 connection(s)
I20260925 00:43:31.890852 14151 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:31.892468 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:31.879497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:40035 (local address 127.6.72.126:45817)
0925 00:43:31.879674 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:31.879681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:31.880559 (+ 878us) server_negotiation.cc:407] Connection header received
0925 00:43:31.881520 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:31.881523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:31.881579 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:31.881661 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:31.883054 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.883951 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.885270 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.885493 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.885609 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:31.888613 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:31.888617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:31.888619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:31.888646 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:31.890360 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:31.890857 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:31.890860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:31.890862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:31.890948 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:31.891290 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:31.891293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:31.891295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:31.891542 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:31.891639 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:31.892182 (+ 543us) server_negotiation.cc:299] Negotiation successful
0925 00:43:31.892349 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 00:43:31.893167 14175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:31.879908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:45817 (local address 127.6.72.66:40035)
0925 00:43:31.880393 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:31.880444 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:31.881305 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:31.882015 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:31.882023 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:31.882425 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:31.882901 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:31.882914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.884136 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.884142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:31.885130 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:31.885140 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.885664 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.885669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:31.885864 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:31.886621 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:31.886637 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:31.888420 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:31.890470 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:31.890476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:31.890489 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:31.890748 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:31.891051 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:31.891055 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:31.891057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:31.891189 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:31.891694 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:31.891699 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:31.891906 (+ 207us) client_negotiation.cc:769] Sending connection context
0925 00:43:31.892049 (+ 143us) client_negotiation.cc:240] Negotiation successful
0925 00:43:31.892351 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":136,"threads_started":1}
I20260925 00:43:31.894447 14282 heartbeater.cc:347] Connected to a master server at 127.6.72.126:45817
I20260925 00:43:31.894613 14282 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:31.895154 14282 heartbeater.cc:511] Master 127.6.72.126:45817 requested a full tablet report, sending...
I20260925 00:43:31.896136 13939 ts_manager.cc:194] Registered new tserver with Master: 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523)
I20260925 00:43:31.896844 13939 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:40035
I20260925 00:43:31.897074 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 14151
I20260925 00:43:31.897198 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:31.897559 6433 external_mini_cluster.cc:1428] Setting key 34e84841c37ca8b9fb16d7d06ba2ed66
I20260925 00:43:31.904691 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:31.896504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:33857 (local address 127.6.72.126:45817)
0925 00:43:31.896618 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:31.896624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:31.897326 (+ 702us) server_negotiation.cc:407] Connection header received
0925 00:43:31.897538 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:31.897541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:31.897587 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:31.897672 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:31.898124 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.898836 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.899665 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.899896 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.900002 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:31.901174 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:31.901178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:31.901180 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:31.901202 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:31.902844 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:31.903359 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:31.903362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:31.903363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:31.903441 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:31.903771 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:31.903773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:31.903774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:31.903983 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:31.904125 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:31.904463 (+ 338us) server_negotiation.cc:299] Negotiation successful
0925 00:43:31.904565 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 00:43:31.904862 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:31.896426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:45817 (local address 127.6.72.66:33857)
0925 00:43:31.897254 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:31.897269 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:31.897392 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:31.897687 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:31.897690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:31.897891 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:31.898002 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:31.898007 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.899001 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.899003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:31.899548 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:31.899555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:31.900006 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:31.900010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:31.900113 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:31.900692 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:31.900703 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:31.901026 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:31.902977 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:31.902982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:31.902987 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:31.903252 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:31.903551 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:31.903557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:31.903562 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:31.903665 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:31.904135 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:31.904141 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:31.904278 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 00:43:31.904409 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 00:43:31.904595 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for oryx/127.6.72.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab.
Entry for principal oryx/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab.
I20260925 00:43:31.927982 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:45817
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab
--principal=oryx/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:37449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:32.041637 14291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:32.041899 14291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:32.041944 14291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:32.045945 14291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:32.046003 14291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:32.046051 14291 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 00:43:32.046085 14291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:32.051344 14291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:37449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab
--principal=oryx/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.72.126:45817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:32.052493 14291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:32.053522 14291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:32.063340 14299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:32.063364 14297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:32.063340 14296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:32.063793 14291 server_base.cc:1034] running on GCE node
I20260925 00:43:32.064298 14291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:32.064975 14291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:32.066175 14291 hybrid_clock.cc:648] HybridClock initialized: now 1790297012066158 us; error 31 us; skew 500 ppm
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297012, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:32.069515 14291 init.cc:377] Logged in from keytab as oryx/127.6.72.67@KRBTEST.COM (short username oryx)
I20260925 00:43:32.070863 14291 webserver.cc:460] Webserver started at http://127.6.72.67:44683/ using document root <none> and password file <none>
I20260925 00:43:32.071563 14291 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:32.071636 14291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:32.071892 14291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:32.074127 14291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "de11d4f98d814ed28e979c8a75e63e2a"
format_stamp: "Formatted at 2026-09-25 00:43:32 on dist-test-slave-w8nh"
server_key: "a7fcce4ae2f449bea8f3b0272b959717"
server_key_iv: "cc5d341c3faedb030daa167f27ff5c07"
server_key_version: "encryptionkey@0"
I20260925 00:43:32.074743 14291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "de11d4f98d814ed28e979c8a75e63e2a"
format_stamp: "Formatted at 2026-09-25 00:43:32 on dist-test-slave-w8nh"
server_key: "a7fcce4ae2f449bea8f3b0272b959717"
server_key_iv: "cc5d341c3faedb030daa167f27ff5c07"
server_key_version: "encryptionkey@0"
I20260925 00:43:32.078754 14291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:43:32.081414 14306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:32.082813 14291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:32.082973 14291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "de11d4f98d814ed28e979c8a75e63e2a"
format_stamp: "Formatted at 2026-09-25 00:43:32 on dist-test-slave-w8nh"
server_key: "a7fcce4ae2f449bea8f3b0272b959717"
server_key_iv: "cc5d341c3faedb030daa167f27ff5c07"
server_key_version: "encryptionkey@0"
I20260925 00:43:32.083101 14291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:32.101002 14291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:32.104204 14291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:32.104475 14291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:32.106041 14291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:32.106117 14291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:32.106194 14291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:32.106238 14291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297012, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:32.117976 14291 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:33151
I20260925 00:43:32.118276 14421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:33151 every 8 connection(s)
I20260925 00:43:32.119185 14291 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:32.121385 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.106807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:35139 (local address 127.6.72.126:45817)
0925 00:43:32.106954 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.106958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.108168 (+ 1210us) server_negotiation.cc:407] Connection header received
0925 00:43:32.109470 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.109478 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.109553 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.109685 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:32.111071 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.111631 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.112883 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.113114 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.113275 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.116935 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:32.116940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:32.116943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:32.116976 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:32.118883 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:32.119566 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:32.119571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:32.119574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:32.119692 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:32.120143 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:32.120146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:32.120148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:32.120418 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:32.120526 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.121055 (+ 529us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.121217 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 00:43:32.122150 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.107456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:45817 (local address 127.6.72.67:35139)
0925 00:43:32.107971 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.108015 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.109094 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.109886 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.109901 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.110266 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:32.110900 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.110919 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.111762 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.111765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.112747 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.112757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.113318 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.113322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.113552 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.114288 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:32.114307 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:32.116735 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:32.119024 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:32.119029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:32.119041 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:32.119434 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:32.119810 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:32.119817 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:32.119819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:32.120005 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:32.120534 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:32.120543 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:32.120810 (+ 267us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.120998 (+ 188us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.121333 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":144,"threads_started":1}
I20260925 00:43:32.123405 14422 heartbeater.cc:347] Connected to a master server at 127.6.72.126:45817
I20260925 00:43:32.123626 14422 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:32.124195 14422 heartbeater.cc:511] Master 127.6.72.126:45817 requested a full tablet report, sending...
I20260925 00:43:32.125238 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 14291
I20260925 00:43:32.125365 13939 ts_manager.cc:194] Registered new tserver with Master: de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151)
I20260925 00:43:32.125384 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:32.125751 6433 external_mini_cluster.cc:1428] Setting key 8dd6e460c8de639482d99a0d01bfbd3d
I20260925 00:43:32.126160 13939 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:35139
I20260925 00:43:32.128788 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:43:32.133967 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.125571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:45817 (local address 127.6.72.67:43625)
0925 00:43:32.125903 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.125917 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.126014 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.126447 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.126450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.126680 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:32.126868 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.126877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.127932 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.127938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.128843 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.128852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.129415 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.129418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.129488 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.130093 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:32.130106 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:32.130431 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:32.132335 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:32.132337 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:32.132339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:32.132539 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:32.132845 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:32.132847 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:32.132848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:32.132931 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:32.133434 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:32.133437 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:32.133520 (+ 83us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.133608 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.133766 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":100,"threads_started":1}
I20260925 00:43:32.134066 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.125642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:43625 (local address 127.6.72.126:45817)
0925 00:43:32.125750 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.125754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.126006 (+ 252us) server_negotiation.cc:407] Connection header received
0925 00:43:32.126156 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.126159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.126247 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.126331 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:32.126994 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.127729 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.128971 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.129292 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.129401 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.130575 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:32.130579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:32.130586 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:32.130619 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:32.132190 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:32.132649 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:32.132655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:32.132660 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:32.132753 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:32.133018 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:32.133024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:32.133028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:32.133332 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:32.133439 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.133701 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.133837 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 00:43:32.155031 13988 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:32.148175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.6.72.126:45817)
0925 00:43:32.148368 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.148371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.148384 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:43:32.148427 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.148429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.148484 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.148554 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:32.149418 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.149940 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.150898 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.151108 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.151192 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.154897 (+ 3705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60618: BlockingRecv error: recv got EOF from 127.0.0.1:60618 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:32.165845 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.156754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60634 (local address 127.6.72.126:45817)
0925 00:43:32.156895 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.156902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.157054 (+ 152us) server_negotiation.cc:407] Connection header received
0925 00:43:32.157234 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.157236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.157322 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.157399 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:32.158028 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.158534 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.159470 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.159674 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.159759 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.162611 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:32.162614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:32.162616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:32.162642 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:32.164137 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:32.164586 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:32.164589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:32.164590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:32.164667 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:32.164949 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:32.164952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:32.164954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:32.165161 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:32.165306 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.165535 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.165633 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 00:43:32.169598 13939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:32.173305 13939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:32.189275 14447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.182545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33151 (local address 127.0.0.1:51942)
0925 00:43:32.183658 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.183684 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.183846 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.184403 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.184409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.184433 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:32.184580 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.184586 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.186122 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.186125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.187708 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.187719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.188680 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.188684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.188792 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.188835 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.188957 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.189096 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1050,"mutex_wait_us":638,"thread_start_us":99,"threads_started":1}
I20260925 00:43:32.189507 14449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.183484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.6.72.66:45523)
0925 00:43:32.183867 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.183880 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.183902 (+ 22us) server_negotiation.cc:407] Connection header received
0925 00:43:32.183979 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.183985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.184170 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.184428 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:32.185094 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.185992 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.187594 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.188563 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.188676 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.188847 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.189264 (+ 417us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.189367 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":88,"threads_started":1}
I20260925 00:43:32.189558 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.182752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.6.72.65:35161)
0925 00:43:32.183359 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.183366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.183954 (+ 588us) server_negotiation.cc:407] Connection header received
0925 00:43:32.184027 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.184032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.184219 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.184400 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:32.184857 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.185955 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.187848 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.188649 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.188737 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.188847 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.189345 (+ 498us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.189430 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":62,"threads_started":1}
I20260925 00:43:32.190076 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.182767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.6.72.67:33151)
0925 00:43:32.183180 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.183193 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.183773 (+ 580us) server_negotiation.cc:407] Connection header received
0925 00:43:32.184016 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.184022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.184219 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.184400 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:32.184701 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.185955 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.187861 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.188539 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.188669 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.188845 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.189882 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.189968 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":98,"threads_started":1}
I20260925 00:43:32.189888 14448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.183335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:45523 (local address 127.0.0.1:34176)
0925 00:43:32.183674 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.183688 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.183817 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.184559 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.184562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.184588 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:32.184968 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.184974 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.186126 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.186132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.187467 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.187477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.188913 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.188916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.189056 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.189098 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.189170 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.189792 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":974,"spinlock_wait_cycles":10112,"thread_start_us":95,"threads_started":1}
I20260925 00:43:32.189975 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.182545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35161 (local address 127.0.0.1:33988)
0925 00:43:32.183610 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.183667 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.183817 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.184559 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.184562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.184590 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:32.184727 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.184733 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.186098 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.186103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.187708 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.187719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.188922 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.188924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.189059 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.189104 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.189170 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.189272 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":154,"spinlock_wait_cycles":13568,"thread_start_us":107,"threads_started":1}
I20260925 00:43:32.193387 14356 tablet_service.cc:1467] Processing CreateTablet for tablet 7b204b286b544a57902042b23ea0c6c0 (DEFAULT_TABLE table=test-table [id=de20ec3d994447d39f0632a8903d395a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:32.193372 14076 tablet_service.cc:1467] Processing CreateTablet for tablet 7b204b286b544a57902042b23ea0c6c0 (DEFAULT_TABLE table=test-table [id=de20ec3d994447d39f0632a8903d395a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:32.194553 14076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b204b286b544a57902042b23ea0c6c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.194553 14356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b204b286b544a57902042b23ea0c6c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.194963 14216 tablet_service.cc:1467] Processing CreateTablet for tablet 7b204b286b544a57902042b23ea0c6c0 (DEFAULT_TABLE table=test-table [id=de20ec3d994447d39f0632a8903d395a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:32.195937 14216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b204b286b544a57902042b23ea0c6c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.202630 14451 tablet_bootstrap.cc:492] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: Bootstrap starting.
I20260925 00:43:32.203095 14450 tablet_bootstrap.cc:492] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: Bootstrap starting.
I20260925 00:43:32.203743 14452 tablet_bootstrap.cc:492] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: Bootstrap starting.
I20260925 00:43:32.205926 14450 tablet_bootstrap.cc:654] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.206012 14451 tablet_bootstrap.cc:654] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.206012 14452 tablet_bootstrap.cc:654] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.206710 14450 log.cc:824] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:32.206835 14452 log.cc:824] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:32.206835 14451 log.cc:824] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:32.208796 14452 tablet_bootstrap.cc:492] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: No bootstrap required, opened a new log
I20260925 00:43:32.208813 14450 tablet_bootstrap.cc:492] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: No bootstrap required, opened a new log
I20260925 00:43:32.208814 14451 tablet_bootstrap.cc:492] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: No bootstrap required, opened a new log
I20260925 00:43:32.209007 14450 ts_tablet_manager.cc:1397] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:43:32.209007 14451 ts_tablet_manager.cc:1397] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 00:43:32.209012 14452 ts_tablet_manager.cc:1397] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 00:43:32.212563 14452 raft_consensus.cc:348] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } }
I20260925 00:43:32.212841 14452 raft_consensus.cc:374] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.212896 14452 raft_consensus.cc:729] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fcad5c64a614a4c9d379852df217d27, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.213199 14451 raft_consensus.cc:348] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } }
I20260925 00:43:32.213455 14451 raft_consensus.cc:374] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.213385 14452 consensus_queue.cc:260] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } }
I20260925 00:43:32.213511 14451 raft_consensus.cc:729] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de11d4f98d814ed28e979c8a75e63e2a, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.213629 14450 raft_consensus.cc:348] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } }
I20260925 00:43:32.213873 14450 raft_consensus.cc:374] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.213927 14450 raft_consensus.cc:729] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e495c846981446c8c6b099b0a1dd2c6, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.213999 14451 consensus_queue.cc:260] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } }
I20260925 00:43:32.214324 14282 heartbeater.cc:503] Master 127.6.72.126:45817 was elected leader, sending a full tablet report...
I20260925 00:43:32.214373 14450 consensus_queue.cc:260] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } }
I20260925 00:43:32.214561 14452 ts_tablet_manager.cc:1428] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 00:43:32.215096 14451 ts_tablet_manager.cc:1428] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:43:32.215178 14450 ts_tablet_manager.cc:1428] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:43:32.215453 14422 heartbeater.cc:503] Master 127.6.72.126:45817 was elected leader, sending a full tablet report...
I20260925 00:43:32.215628 14142 heartbeater.cc:503] Master 127.6.72.126:45817 was elected leader, sending a full tablet report...
I20260925 00:43:32.228616 14456 raft_consensus.cc:482] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:32.228866 14456 raft_consensus.cc:504] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } }
I20260925 00:43:32.230062 14456 leader_election.cc:288] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161), de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151)
I20260925 00:43:32.234203 14175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.230314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35161 (local address 127.6.72.66:49791)
0925 00:43:32.230561 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.230579 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.230692 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.231036 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.231039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.231059 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:32.231197 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.231205 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.232052 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.232057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.233130 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.233141 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.233827 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.233830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.233952 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.233972 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.234029 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.234089 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":28}
I20260925 00:43:32.234274 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.230385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:49791 (local address 127.6.72.65:35161)
0925 00:43:32.230508 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.230513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.230657 (+ 144us) server_negotiation.cc:407] Connection header received
0925 00:43:32.230851 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.230854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.230915 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.230982 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:32.231314 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.231914 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.233280 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.233685 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.233784 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.233838 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.234070 (+ 232us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.234150 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 00:43:32.234965 14096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b204b286b544a57902042b23ea0c6c0" candidate_uuid: "4fcad5c64a614a4c9d379852df217d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e495c846981446c8c6b099b0a1dd2c6" is_pre_election: true
I20260925 00:43:32.235069 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.230619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:47997 (local address 127.6.72.67:33151)
0925 00:43:32.230895 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.230899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.231229 (+ 330us) server_negotiation.cc:407] Connection header received
0925 00:43:32.231311 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.231314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.231363 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.231443 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:32.232366 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.232839 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.233956 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.234549 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.234656 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.234720 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.234921 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.234979 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260925 00:43:32.235162 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.230314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33151 (local address 127.6.72.66:47997)
0925 00:43:32.231115 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.231128 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.231219 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.231514 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.231517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.231536 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:32.232213 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.232222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.232947 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.232950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.233820 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.233829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.234701 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.234705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.234798 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.234822 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.234899 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.234963 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":354,"thread_start_us":162,"threads_started":1}
I20260925 00:43:32.235332 14096 raft_consensus.cc:2454] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fcad5c64a614a4c9d379852df217d27 in term 0.
I20260925 00:43:32.235724 14376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b204b286b544a57902042b23ea0c6c0" candidate_uuid: "4fcad5c64a614a4c9d379852df217d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de11d4f98d814ed28e979c8a75e63e2a" is_pre_election: true
I20260925 00:43:32.235832 14169 leader_election.cc:302] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fcad5c64a614a4c9d379852df217d27, 8e495c846981446c8c6b099b0a1dd2c6; no voters:
I20260925 00:43:32.236049 14376 raft_consensus.cc:2454] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fcad5c64a614a4c9d379852df217d27 in term 0.
I20260925 00:43:32.236187 14456 raft_consensus.cc:2781] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:32.236337 14456 raft_consensus.cc:482] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:32.236410 14456 raft_consensus.cc:3037] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:32.237453 14456 raft_consensus.cc:504] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } }
I20260925 00:43:32.237831 14456 leader_election.cc:288] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [CANDIDATE]: Term 1 election: Requested vote from peers 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161), de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151)
I20260925 00:43:32.238243 14096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b204b286b544a57902042b23ea0c6c0" candidate_uuid: "4fcad5c64a614a4c9d379852df217d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e495c846981446c8c6b099b0a1dd2c6"
I20260925 00:43:32.238292 14376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b204b286b544a57902042b23ea0c6c0" candidate_uuid: "4fcad5c64a614a4c9d379852df217d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de11d4f98d814ed28e979c8a75e63e2a"
I20260925 00:43:32.238363 14096 raft_consensus.cc:3037] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:32.238441 14376 raft_consensus.cc:3037] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:32.239490 14096 raft_consensus.cc:2454] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fcad5c64a614a4c9d379852df217d27 in term 1.
I20260925 00:43:32.239513 14376 raft_consensus.cc:2454] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fcad5c64a614a4c9d379852df217d27 in term 1.
I20260925 00:43:32.239897 14169 leader_election.cc:302] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fcad5c64a614a4c9d379852df217d27, 8e495c846981446c8c6b099b0a1dd2c6; no voters:
I20260925 00:43:32.240104 14456 raft_consensus.cc:2781] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:32.240393 14456 raft_consensus.cc:686] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 1 LEADER]: Becoming Leader. State: Replica: 4fcad5c64a614a4c9d379852df217d27, State: Running, Role: LEADER
I20260925 00:43:32.240782 14456 consensus_queue.cc:237] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } }
I20260925 00:43:32.244196 13938 catalog_manager.cc:5495] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66). New cstate: current_term: 1 leader_uuid: "4fcad5c64a614a4c9d379852df217d27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } health_report { overall_health: HEALTHY } } }
I20260925 00:43:32.268957 14449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.265383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.6.72.66:45523)
0925 00:43:32.265530 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.265533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.265546 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:43:32.265623 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.265625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.265679 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.265775 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:32.266001 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.266502 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.267299 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.267540 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.267623 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.267975 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:32.268541 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:32.268643 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.268754 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.268829 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 00:43:32.275349 14376 raft_consensus.cc:1261] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Refusing update from remote peer 4fcad5c64a614a4c9d379852df217d27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:32.275349 14096 raft_consensus.cc:1261] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Refusing update from remote peer 4fcad5c64a614a4c9d379852df217d27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:32.276120 14460 consensus_queue.cc:1035] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:32.276358 14456 consensus_queue.cc:1035] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:32.287111 14465 mvcc.cc:205] Tried to move back new op lower bound from 7333056562271940608 to 7333056562140954624. Current Snapshot: MvccSnapshot[applied={T|T < 7333056562271940608}]
I20260925 00:43:32.290180 14464 mvcc.cc:205] Tried to move back new op lower bound from 7333056562271940608 to 7333056562140954624. Current Snapshot: MvccSnapshot[applied={T|T < 7333056562271940608 or (T in {7333056562271940608})}]
I20260925 00:43:32.293843 13939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:32.294137 13939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:32.296962 13939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453: Sending DeleteTablet for 3 replicas of tablet 7b204b286b544a57902042b23ea0c6c0
I20260925 00:43:32.297894 14216 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b204b286b544a57902042b23ea0c6c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:32 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:34176
I20260925 00:43:32.297894 14356 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b204b286b544a57902042b23ea0c6c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:32 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:51942
I20260925 00:43:32.298475 14473 tablet_replica.cc:330] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: stopping tablet replica
I20260925 00:43:32.298641 14076 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b204b286b544a57902042b23ea0c6c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:32 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:33988
I20260925 00:43:32.298964 14473 raft_consensus.cc:2229] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:32.299187 14475 tablet_replica.cc:330] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: stopping tablet replica
I20260925 00:43:32.299366 13939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 00:43:32.299476 14475 raft_consensus.cc:2229] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:32.299753 14475 raft_consensus.cc:2258] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 00:43:32.299824 13939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:32.300803 14473 raft_consensus.cc:2258] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:32.301285 14475 ts_tablet_manager.cc:1905] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:32.302457 14473 ts_tablet_manager.cc:1905] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:32.304136 14475 ts_tablet_manager.cc:1918] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:32.304277 14475 log.cc:1196] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/7b204b286b544a57902042b23ea0c6c0
I20260925 00:43:32.304607 14475 ts_tablet_manager.cc:1939] T 7b204b286b544a57902042b23ea0c6c0 P 8e495c846981446c8c6b099b0a1dd2c6: Deleting consensus metadata
I20260925 00:43:32.305680 14076 tablet_service.cc:1467] Processing CreateTablet for tablet 738096bb6cac42df84dbf378db6f848a (DEFAULT_TABLE table=test-table [id=bf4eafa2ef7049dc91d4e7e8833c66ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:32.305953 14076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 738096bb6cac42df84dbf378db6f848a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.305979 14356 tablet_service.cc:1467] Processing CreateTablet for tablet 738096bb6cac42df84dbf378db6f848a (DEFAULT_TABLE table=test-table [id=bf4eafa2ef7049dc91d4e7e8833c66ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:32.306248 14356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 738096bb6cac42df84dbf378db6f848a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.308419 14450 tablet_bootstrap.cc:492] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: Bootstrap starting.
I20260925 00:43:32.308774 14216 tablet_service.cc:1467] Processing CreateTablet for tablet 738096bb6cac42df84dbf378db6f848a (DEFAULT_TABLE table=test-table [id=bf4eafa2ef7049dc91d4e7e8833c66ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:32.309033 14216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 738096bb6cac42df84dbf378db6f848a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.309186 14451 tablet_bootstrap.cc:492] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: Bootstrap starting.
I20260925 00:43:32.309492 14450 tablet_bootstrap.cc:654] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.310218 14451 tablet_bootstrap.cc:654] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.311321 14450 tablet_bootstrap.cc:492] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: No bootstrap required, opened a new log
I20260925 00:43:32.311393 14452 tablet_bootstrap.cc:492] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: Bootstrap starting.
I20260925 00:43:32.311401 14450 ts_tablet_manager.cc:1397] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:32.311740 13925 catalog_manager.cc:4841] TS 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161): tablet 7b204b286b544a57902042b23ea0c6c0 (table test-table [id=de20ec3d994447d39f0632a8903d395a]) successfully deleted
I20260925 00:43:32.312134 14450 raft_consensus.cc:348] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.312268 14450 raft_consensus.cc:374] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.312302 14450 raft_consensus.cc:729] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e495c846981446c8c6b099b0a1dd2c6, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.312410 14452 tablet_bootstrap.cc:654] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.312424 14450 consensus_queue.cc:260] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.312603 14473 ts_tablet_manager.cc:1918] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:32.312704 14450 ts_tablet_manager.cc:1428] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:32.312706 14473 log.cc:1196] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/7b204b286b544a57902042b23ea0c6c0
I20260925 00:43:32.313045 14473 ts_tablet_manager.cc:1939] T 7b204b286b544a57902042b23ea0c6c0 P 4fcad5c64a614a4c9d379852df217d27: Deleting consensus metadata
I20260925 00:43:32.314283 14451 tablet_bootstrap.cc:492] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: No bootstrap required, opened a new log
I20260925 00:43:32.314373 14451 ts_tablet_manager.cc:1397] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 00:43:32.314394 13925 catalog_manager.cc:4841] TS 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523): tablet 7b204b286b544a57902042b23ea0c6c0 (table test-table [id=de20ec3d994447d39f0632a8903d395a]) successfully deleted
I20260925 00:43:32.314842 14451 raft_consensus.cc:348] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.315013 14451 raft_consensus.cc:374] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.315047 14451 raft_consensus.cc:729] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de11d4f98d814ed28e979c8a75e63e2a, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.315176 14451 consensus_queue.cc:260] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.315421 14451 ts_tablet_manager.cc:1428] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:32.315635 14452 tablet_bootstrap.cc:492] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: No bootstrap required, opened a new log
I20260925 00:43:32.315747 14452 ts_tablet_manager.cc:1397] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 00:43:32.314843 14474 tablet_replica.cc:330] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: stopping tablet replica
I20260925 00:43:32.316265 14452 raft_consensus.cc:348] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.316388 14452 raft_consensus.cc:374] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.316423 14452 raft_consensus.cc:729] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fcad5c64a614a4c9d379852df217d27, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.316424 14474 raft_consensus.cc:2229] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:32.316536 14452 consensus_queue.cc:260] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.316751 14474 raft_consensus.cc:2258] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:32.317999 14452 ts_tablet_manager.cc:1428] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 00:43:32.318459 14474 ts_tablet_manager.cc:1905] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:32.321278 14474 ts_tablet_manager.cc:1918] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:32.321384 14474 log.cc:1196] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/7b204b286b544a57902042b23ea0c6c0
I20260925 00:43:32.321684 14474 ts_tablet_manager.cc:1939] T 7b204b286b544a57902042b23ea0c6c0 P de11d4f98d814ed28e979c8a75e63e2a: Deleting consensus metadata
I20260925 00:43:32.323201 13923 catalog_manager.cc:4841] TS de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151): tablet 7b204b286b544a57902042b23ea0c6c0 (table test-table [id=de20ec3d994447d39f0632a8903d395a]) successfully deleted
W20260925 00:43:32.370234 14423 tablet.cc:2369] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:43:32.392186 14283 tablet.cc:2369] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:43:32.416464 14143 tablet.cc:2369] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:43:32.534137 14458 raft_consensus.cc:482] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:32.534358 14458 raft_consensus.cc:504] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.535992 14458 leader_election.cc:288] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523), de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151)
I20260925 00:43:32.541915 14449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.536645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:39915 (local address 127.6.72.66:45523)
0925 00:43:32.536785 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.536791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.536809 (+ 18us) server_negotiation.cc:407] Connection header received
0925 00:43:32.536868 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.536871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.536922 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.536998 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:32.537472 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.538284 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.540418 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.541050 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.541167 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.541245 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.541683 (+ 438us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.541778 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 00:43:32.541910 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.536462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:50797 (local address 127.6.72.67:33151)
0925 00:43:32.536639 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.536643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.536830 (+ 187us) server_negotiation.cc:407] Connection header received
0925 00:43:32.536927 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.536929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.536979 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.537050 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:32.537472 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.538284 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.540418 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.541049 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.541167 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.541243 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.541683 (+ 440us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.541777 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 00:43:32.541955 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.536283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:45523 (local address 127.6.72.65:39915)
0925 00:43:32.536560 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.536576 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.536692 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.537006 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.537008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.537048 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:32.537316 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.537323 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.538437 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.538442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.540266 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.540280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.541254 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.541259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.541457 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.541512 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.541604 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.541715 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":249,"spinlock_wait_cycles":28288,"thread_start_us":113,"threads_started":1}
I20260925 00:43:32.541953 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.536328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33151 (local address 127.6.72.65:50797)
0925 00:43:32.536711 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.536727 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.536838 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.537090 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.537093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.537130 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:32.537307 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.537317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.538437 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.538442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.540266 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.540280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.541254 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.541259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.541463 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.541508 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.541650 (+ 142us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.541715 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":263,"spinlock_wait_cycles":896,"thread_start_us":125,"threads_started":1}
I20260925 00:43:32.542697 14376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738096bb6cac42df84dbf378db6f848a" candidate_uuid: "8e495c846981446c8c6b099b0a1dd2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de11d4f98d814ed28e979c8a75e63e2a" is_pre_election: true
I20260925 00:43:32.542697 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738096bb6cac42df84dbf378db6f848a" candidate_uuid: "8e495c846981446c8c6b099b0a1dd2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fcad5c64a614a4c9d379852df217d27" is_pre_election: true
I20260925 00:43:32.542881 14376 raft_consensus.cc:2454] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e495c846981446c8c6b099b0a1dd2c6 in term 0.
I20260925 00:43:32.542939 14236 raft_consensus.cc:2454] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e495c846981446c8c6b099b0a1dd2c6 in term 0.
I20260925 00:43:32.543331 14027 leader_election.cc:302] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e495c846981446c8c6b099b0a1dd2c6, de11d4f98d814ed28e979c8a75e63e2a; no voters:
I20260925 00:43:32.543638 14458 raft_consensus.cc:2781] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:32.543735 14458 raft_consensus.cc:482] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:32.543766 14458 raft_consensus.cc:3037] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:32.545397 14458 raft_consensus.cc:504] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.545846 14458 leader_election.cc:288] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [CANDIDATE]: Term 1 election: Requested vote from peers 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523), de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151)
I20260925 00:43:32.546222 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738096bb6cac42df84dbf378db6f848a" candidate_uuid: "8e495c846981446c8c6b099b0a1dd2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fcad5c64a614a4c9d379852df217d27"
I20260925 00:43:32.546222 14376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738096bb6cac42df84dbf378db6f848a" candidate_uuid: "8e495c846981446c8c6b099b0a1dd2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de11d4f98d814ed28e979c8a75e63e2a"
I20260925 00:43:32.546347 14236 raft_consensus.cc:3037] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:32.546371 14376 raft_consensus.cc:3037] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:32.547310 14376 raft_consensus.cc:2454] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e495c846981446c8c6b099b0a1dd2c6 in term 1.
I20260925 00:43:32.547312 14236 raft_consensus.cc:2454] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e495c846981446c8c6b099b0a1dd2c6 in term 1.
I20260925 00:43:32.547647 14027 leader_election.cc:302] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e495c846981446c8c6b099b0a1dd2c6, de11d4f98d814ed28e979c8a75e63e2a; no voters:
I20260925 00:43:32.547942 14458 raft_consensus.cc:2781] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:32.548183 14458 raft_consensus.cc:686] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 LEADER]: Becoming Leader. State: Replica: 8e495c846981446c8c6b099b0a1dd2c6, State: Running, Role: LEADER
I20260925 00:43:32.548491 14458 consensus_queue.cc:237] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.550834 13937 catalog_manager.cc:5495] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65). New cstate: current_term: 1 leader_uuid: "8e495c846981446c8c6b099b0a1dd2c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } health_report { overall_health: UNKNOWN } } }
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297011, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.126@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:32.630868 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.622062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:45817 (local address 127.0.0.1:60650)
0925 00:43:32.622385 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.622397 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.622529 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.622778 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.622781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.623193 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:32.623333 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.623342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.624088 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.624093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.624777 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.624787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.625215 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.625218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.625302 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.625930 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:32.625943 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:32.627590 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:32.629460 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:32.629470 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:32.629475 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:32.629706 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:32.629946 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:32.629948 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:32.629949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:32.630022 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:32.630367 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:32.630373 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:32.630475 (+ 102us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.630578 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.630703 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":128,"threads_started":1}
I20260925 00:43:32.630883 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.622120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.6.72.126:45817)
0925 00:43:32.622292 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.622297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.622467 (+ 170us) server_negotiation.cc:407] Connection header received
0925 00:43:32.622622 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.622625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.622667 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.622761 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:32.623448 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.623972 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.624892 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.625099 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.625218 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.627767 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:32.627770 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:32.627772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:32.627794 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:32.629358 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:32.629795 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:32.629798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:32.629799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:32.629870 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:32.630094 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:32.630096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:32.630097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:32.630282 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:32.630355 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.630632 (+ 277us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.630759 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":46}
I20260925 00:43:32.635416 13937 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:60650:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 00:43:32.636336 13937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:32.641337 14216 tablet_service.cc:1467] Processing CreateTablet for tablet 2fdbb252f58c47988fde792aebc4e091 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c58706f85dc4c0ba0ecf5ee2f1a87b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 00:43:32.641338 14076 tablet_service.cc:1467] Processing CreateTablet for tablet 2fdbb252f58c47988fde792aebc4e091 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c58706f85dc4c0ba0ecf5ee2f1a87b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 00:43:32.641436 14356 tablet_service.cc:1467] Processing CreateTablet for tablet 2fdbb252f58c47988fde792aebc4e091 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c58706f85dc4c0ba0ecf5ee2f1a87b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 00:43:32.641760 14076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fdbb252f58c47988fde792aebc4e091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.641764 14216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fdbb252f58c47988fde792aebc4e091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.641785 14356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fdbb252f58c47988fde792aebc4e091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.644416 14452 tablet_bootstrap.cc:492] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: Bootstrap starting.
I20260925 00:43:32.645351 14452 tablet_bootstrap.cc:654] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.645404 14451 tablet_bootstrap.cc:492] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: Bootstrap starting.
I20260925 00:43:32.646281 14451 tablet_bootstrap.cc:654] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.646389 14450 tablet_bootstrap.cc:492] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: Bootstrap starting.
I20260925 00:43:32.646903 14452 tablet_bootstrap.cc:492] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: No bootstrap required, opened a new log
I20260925 00:43:32.646997 14452 ts_tablet_manager.cc:1397] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:32.647331 14450 tablet_bootstrap.cc:654] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.647441 14452 raft_consensus.cc:348] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.647588 14452 raft_consensus.cc:374] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.647620 14452 raft_consensus.cc:729] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fcad5c64a614a4c9d379852df217d27, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.647775 14452 consensus_queue.cc:260] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.648056 14452 ts_tablet_manager.cc:1428] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:32.648296 14469 tablet_replica.cc:439] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } }
I20260925 00:43:32.648442 14469 tablet_replica.cc:442] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:32.648998 14450 tablet_bootstrap.cc:492] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: No bootstrap required, opened a new log
I20260925 00:43:32.648998 14451 tablet_bootstrap.cc:492] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: No bootstrap required, opened a new log
I20260925 00:43:32.649089 14450 ts_tablet_manager.cc:1397] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:32.649089 14451 ts_tablet_manager.cc:1397] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 00:43:32.649519 14450 raft_consensus.cc:348] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.649519 14451 raft_consensus.cc:348] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.649636 14451 raft_consensus.cc:374] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.649645 14450 raft_consensus.cc:374] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.649685 14451 raft_consensus.cc:729] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de11d4f98d814ed28e979c8a75e63e2a, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.649685 14450 raft_consensus.cc:729] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e495c846981446c8c6b099b0a1dd2c6, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.649814 14450 consensus_queue.cc:260] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.649814 14451 consensus_queue.cc:260] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.650075 14450 ts_tablet_manager.cc:1428] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 00:43:32.650075 14451 ts_tablet_manager.cc:1428] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:32.650172 14457 tablet_replica.cc:439] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } }
I20260925 00:43:32.650336 14457 tablet_replica.cc:442] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:32.650523 14458 tablet_replica.cc:439] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } }
I20260925 00:43:32.650655 14458 tablet_replica.cc:442] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:32.710518 14457 raft_consensus.cc:482] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:32.710667 14457 raft_consensus.cc:504] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.711859 14457 leader_election.cc:288] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161), 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523)
I20260925 00:43:32.716109 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.712164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35161 (local address 127.6.72.67:53693)
0925 00:43:32.712483 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.712499 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.712641 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.712941 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.712944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.712964 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:32.713093 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.713099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.713950 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.713956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.714945 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.714959 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.715614 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.715617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.715718 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.715750 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.715847 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.715929 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":27,"thread_start_us":114,"threads_started":1}
I20260925 00:43:32.716104 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.712314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:53693 (local address 127.6.72.65:35161)
0925 00:43:32.712467 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.712472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.712605 (+ 133us) server_negotiation.cc:407] Connection header received
0925 00:43:32.712762 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.712766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.712828 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.712928 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:32.713205 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.713841 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.715076 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.715485 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.715610 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.715681 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.715861 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.715999 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:43:32.716192 14449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.712714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:33871 (local address 127.6.72.66:45523)
0925 00:43:32.712836 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.712840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.712900 (+ 60us) server_negotiation.cc:407] Connection header received
0925 00:43:32.713013 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.713016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.713068 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.713156 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:32.713412 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.713962 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.715174 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.715573 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.715691 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.715752 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.715974 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.716070 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 00:43:32.716605 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.712598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:45523 (local address 127.6.72.67:33871)
0925 00:43:32.712839 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.712852 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.712932 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.713163 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.713165 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.713180 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:32.713303 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.713308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.714073 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.714077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.715058 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.715067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.715694 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.715697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.715792 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.715814 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.716372 (+ 558us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.716462 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":37,"thread_start_us":67,"threads_started":1}
I20260925 00:43:32.716634 14096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fdbb252f58c47988fde792aebc4e091" candidate_uuid: "de11d4f98d814ed28e979c8a75e63e2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e495c846981446c8c6b099b0a1dd2c6" is_pre_election: true
I20260925 00:43:32.716782 14096 raft_consensus.cc:2454] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de11d4f98d814ed28e979c8a75e63e2a in term 0.
I20260925 00:43:32.717054 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fdbb252f58c47988fde792aebc4e091" candidate_uuid: "de11d4f98d814ed28e979c8a75e63e2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fcad5c64a614a4c9d379852df217d27" is_pre_election: true
I20260925 00:43:32.717149 14309 leader_election.cc:302] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e495c846981446c8c6b099b0a1dd2c6, de11d4f98d814ed28e979c8a75e63e2a; no voters:
I20260925 00:43:32.717223 14236 raft_consensus.cc:2454] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de11d4f98d814ed28e979c8a75e63e2a in term 0.
I20260925 00:43:32.717445 14457 raft_consensus.cc:2781] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:32.717507 14457 raft_consensus.cc:482] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:32.717554 14457 raft_consensus.cc:3037] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:32.718528 14457 raft_consensus.cc:504] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.718860 14457 leader_election.cc:288] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [CANDIDATE]: Term 1 election: Requested vote from peers 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161), 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523)
I20260925 00:43:32.719164 14096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fdbb252f58c47988fde792aebc4e091" candidate_uuid: "de11d4f98d814ed28e979c8a75e63e2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e495c846981446c8c6b099b0a1dd2c6"
I20260925 00:43:32.719296 14096 raft_consensus.cc:3037] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:32.719316 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fdbb252f58c47988fde792aebc4e091" candidate_uuid: "de11d4f98d814ed28e979c8a75e63e2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fcad5c64a614a4c9d379852df217d27"
I20260925 00:43:32.719448 14236 raft_consensus.cc:3037] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:32.720306 14096 raft_consensus.cc:2454] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de11d4f98d814ed28e979c8a75e63e2a in term 1.
I20260925 00:43:32.720593 14236 raft_consensus.cc:2454] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de11d4f98d814ed28e979c8a75e63e2a in term 1.
I20260925 00:43:32.720602 14309 leader_election.cc:302] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e495c846981446c8c6b099b0a1dd2c6, de11d4f98d814ed28e979c8a75e63e2a; no voters:
I20260925 00:43:32.720790 14457 raft_consensus.cc:2781] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:32.720965 14457 raft_consensus.cc:686] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [term 1 LEADER]: Becoming Leader. State: Replica: de11d4f98d814ed28e979c8a75e63e2a, State: Running, Role: LEADER
I20260925 00:43:32.721287 14457 consensus_queue.cc:237] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.722725 14494 tablet_replica.cc:439] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: TxnStatusTablet state changed. Reason: New leader de11d4f98d814ed28e979c8a75e63e2a. Latest consensus state: current_term: 1 leader_uuid: "de11d4f98d814ed28e979c8a75e63e2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } }
I20260925 00:43:32.722880 14494 tablet_replica.cc:442] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: This TxnStatusTablet replica's current role is: LEADER
I20260925 00:43:32.723163 14496 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 00:43:32.723321 14496 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 00:43:32.723969 13939 catalog_manager.cc:5495] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a reported cstate change: term changed from 0 to 1, leader changed from <none> to de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67). New cstate: current_term: 1 leader_uuid: "de11d4f98d814ed28e979c8a75e63e2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } health_report { overall_health: HEALTHY } } }
I20260925 00:43:32.751245 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.747362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33151 (local address 127.0.0.1:51952)
0925 00:43:32.747621 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.747638 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.747752 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.748079 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.748082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.748092 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:43:32.748210 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.748215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.748987 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.748990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.749607 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.749615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.750060 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.750064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.750161 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.750210 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:43:32.750924 (+ 714us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:43:32.750942 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.751012 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.751120 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260925 00:43:32.751246 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.747509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51952 (local address 127.6.72.67:33151)
0925 00:43:32.747650 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.747657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.747677 (+ 20us) server_negotiation.cc:407] Connection header received
0925 00:43:32.747910 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.747913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.747965 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.748064 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:32.748345 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.748867 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.749724 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.749947 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.750058 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.750363 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:32.750820 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:32.750947 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.751021 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.751075 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 00:43:32.754185 14096 raft_consensus.cc:1261] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Refusing update from remote peer de11d4f98d814ed28e979c8a75e63e2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:32.754298 14236 raft_consensus.cc:1261] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Refusing update from remote peer de11d4f98d814ed28e979c8a75e63e2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:32.754712 14494 consensus_queue.cc:1035] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:32.754977 14457 consensus_queue.cc:1035] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:32.756500 14458 tablet_replica.cc:439] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: TxnStatusTablet state changed. Reason: New leader de11d4f98d814ed28e979c8a75e63e2a. Latest consensus state: current_term: 1 leader_uuid: "de11d4f98d814ed28e979c8a75e63e2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } }
I20260925 00:43:32.756625 14458 tablet_replica.cc:442] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:32.756690 14469 tablet_replica.cc:439] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: TxnStatusTablet state changed. Reason: New leader de11d4f98d814ed28e979c8a75e63e2a. Latest consensus state: current_term: 1 leader_uuid: "de11d4f98d814ed28e979c8a75e63e2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } }
I20260925 00:43:32.756799 14469 tablet_replica.cc:442] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:32.758836 14494 tablet_replica.cc:439] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de11d4f98d814ed28e979c8a75e63e2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } }
I20260925 00:43:32.758977 14494 tablet_replica.cc:442] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: This TxnStatusTablet replica's current role is: LEADER
I20260925 00:43:32.760298 14463 mvcc.cc:205] Tried to move back new op lower bound from 7333056564235726848 to 7333056564109578240. Current Snapshot: MvccSnapshot[applied={T|T < 7333056564235726848}]
I20260925 00:43:32.760424 14469 tablet_replica.cc:439] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de11d4f98d814ed28e979c8a75e63e2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } }
I20260925 00:43:32.760537 14469 tablet_replica.cc:442] T 2fdbb252f58c47988fde792aebc4e091 P 4fcad5c64a614a4c9d379852df217d27: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:32.763401 14494 tablet_replica.cc:439] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de11d4f98d814ed28e979c8a75e63e2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } }
I20260925 00:43:32.763541 14494 tablet_replica.cc:442] T 2fdbb252f58c47988fde792aebc4e091 P de11d4f98d814ed28e979c8a75e63e2a: This TxnStatusTablet replica's current role is: LEADER
I20260925 00:43:32.765004 14458 tablet_replica.cc:439] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de11d4f98d814ed28e979c8a75e63e2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } } }
I20260925 00:43:32.765139 14458 tablet_replica.cc:442] T 2fdbb252f58c47988fde792aebc4e091 P 8e495c846981446c8c6b099b0a1dd2c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 00:43:32.769258 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.765127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33990 (local address 127.6.72.65:35161)
0925 00:43:32.765544 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.765549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.765562 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:43:32.765640 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.765643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.765698 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.765809 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:32.766117 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.766884 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.767737 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.767979 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.768089 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.768403 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:32.768859 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:32.768994 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.769054 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.769118 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349}
I20260925 00:43:32.780364 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.776806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:40235 (local address 127.6.72.67:33151)
0925 00:43:32.776936 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.776941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.777086 (+ 145us) server_negotiation.cc:407] Connection header received
0925 00:43:32.777210 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.777213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.777269 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.777350 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:32.777578 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.778332 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.779212 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.779547 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.779648 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.779914 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:32.779998 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:32.780068 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.780144 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.780198 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 00:43:32.780390 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.776699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33151 (local address 127.6.72.65:40235)
0925 00:43:32.777002 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.777014 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.777109 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.777368 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.777371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.777376 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:43:32.777479 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.777484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.778461 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.778464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.779091 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.779100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.779670 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.779672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.779742 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.779787 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:43:32.780081 (+ 294us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:43:32.780085 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.780134 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.780201 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":119,"threads_started":1}
I20260925 00:43:32.787482 14376 raft_consensus.cc:1261] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Refusing update from remote peer 8e495c846981446c8c6b099b0a1dd2c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:32.787602 14236 raft_consensus.cc:1261] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Refusing update from remote peer 8e495c846981446c8c6b099b0a1dd2c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:32.787943 14458 consensus_queue.cc:1035] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:32.788286 14481 consensus_queue.cc:1035] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:32.812928 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.805026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:37119 (local address 127.6.72.65:35161)
0925 00:43:32.805131 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.805136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.805678 (+ 542us) server_negotiation.cc:407] Connection header received
0925 00:43:32.805729 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.805732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.805782 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.805852 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:32.806609 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.807317 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.811207 (+ 3890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.811523 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.811611 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.812274 (+ 663us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:32.812366 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:32.812428 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.812714 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.812792 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 00:43:32.812989 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.804957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35161 (local address 127.6.72.67:37119)
0925 00:43:32.805393 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:32.805405 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:32.805528 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:32.806339 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:32.806342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:32.806353 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:43:32.806484 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.806493 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.809665 (+ 3172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.809668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.811074 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:32.811085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.811907 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.811911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:32.812007 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.812055 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:43:32.812493 (+ 438us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:43:32.812498 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 00:43:32.812698 (+ 200us) client_negotiation.cc:240] Negotiation successful
0925 00:43:32.812847 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":118,"threads_started":1}
I20260925 00:43:32.841391 13939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:32.841574 13939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:32.844079 13939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453: Sending DeleteTablet for 3 replicas of tablet 738096bb6cac42df84dbf378db6f848a
I20260925 00:43:32.844898 14356 tablet_service.cc:1514] Processing DeleteTablet for tablet 738096bb6cac42df84dbf378db6f848a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:32 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:51942
I20260925 00:43:32.844982 14076 tablet_service.cc:1514] Processing DeleteTablet for tablet 738096bb6cac42df84dbf378db6f848a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:32 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:33988
I20260925 00:43:32.845263 14216 tablet_service.cc:1514] Processing DeleteTablet for tablet 738096bb6cac42df84dbf378db6f848a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:32 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:34176
I20260925 00:43:32.845736 14513 tablet_replica.cc:330] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: stopping tablet replica
I20260925 00:43:32.845906 14513 raft_consensus.cc:2229] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:32.846299 14513 raft_consensus.cc:2258] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:32.847060 14513 ts_tablet_manager.cc:1905] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:32.850406 14513 ts_tablet_manager.cc:1918] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:32.850564 14513 log.cc:1196] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/738096bb6cac42df84dbf378db6f848a
I20260925 00:43:32.850683 14512 tablet_replica.cc:330] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: stopping tablet replica
I20260925 00:43:32.850826 14512 raft_consensus.cc:2229] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:32.850977 14512 raft_consensus.cc:2258] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:32.851104 14513 ts_tablet_manager.cc:1939] T 738096bb6cac42df84dbf378db6f848a P 8e495c846981446c8c6b099b0a1dd2c6: Deleting consensus metadata
I20260925 00:43:32.851223 14514 tablet_replica.cc:330] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: stopping tablet replica
I20260925 00:43:32.851392 14514 raft_consensus.cc:2229] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:32.851521 14514 raft_consensus.cc:2258] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:32.852079 13925 catalog_manager.cc:4841] TS 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161): tablet 738096bb6cac42df84dbf378db6f848a (table test-table [id=bf4eafa2ef7049dc91d4e7e8833c66ce]) successfully deleted
I20260925 00:43:32.852157 14512 ts_tablet_manager.cc:1905] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:32.855181 14512 ts_tablet_manager.cc:1918] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:32.855273 14512 log.cc:1196] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/738096bb6cac42df84dbf378db6f848a
I20260925 00:43:32.856258 14514 ts_tablet_manager.cc:1905] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:32.859067 14514 ts_tablet_manager.cc:1918] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:32.859153 14514 log.cc:1196] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/738096bb6cac42df84dbf378db6f848a
I20260925 00:43:32.859404 14514 ts_tablet_manager.cc:1939] T 738096bb6cac42df84dbf378db6f848a P 4fcad5c64a614a4c9d379852df217d27: Deleting consensus metadata
I20260925 00:43:32.859697 14512 ts_tablet_manager.cc:1939] T 738096bb6cac42df84dbf378db6f848a P de11d4f98d814ed28e979c8a75e63e2a: Deleting consensus metadata
I20260925 00:43:32.860512 13923 catalog_manager.cc:4841] TS de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151): tablet 738096bb6cac42df84dbf378db6f848a (table test-table [id=bf4eafa2ef7049dc91d4e7e8833c66ce]) successfully deleted
I20260925 00:43:32.861361 13925 catalog_manager.cc:4841] TS 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523): tablet 738096bb6cac42df84dbf378db6f848a (table test-table [id=bf4eafa2ef7049dc91d4e7e8833c66ce]) successfully deleted
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 00:43:32.889426 13988 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:32.881513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60654 (local address 127.6.72.126:45817)
0925 00:43:32.881642 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.881647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.881931 (+ 284us) server_negotiation.cc:407] Connection header received
0925 00:43:32.882058 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.882062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.882125 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.882215 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:32.883073 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.883903 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.885117 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.885469 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.885576 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.889290 (+ 3714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60654: BlockingRecv error: recv got EOF from 127.0.0.1:60654 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 00:43:32 dist-test-slave-w8nh krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM
I20260925 00:43:32.902444 13988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:32.891688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.6.72.126:45817)
0925 00:43:32.891801 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:32.891805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:32.892235 (+ 430us) server_negotiation.cc:407] Connection header received
0925 00:43:32.892465 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:32.892471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:32.892534 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:32.892599 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:32.893455 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.894285 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.895524 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:32.895868 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:32.895939 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:32.898964 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:32.898969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:32.898974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:32.899008 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:32.900651 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:32.901125 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:32.901129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:32.901130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:32.901224 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:32.901489 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:32.901492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:32.901494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:32.901697 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:32.901795 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:32.902143 (+ 348us) server_negotiation.cc:299] Negotiation successful
0925 00:43:32.902275 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 00:43:32.905867 13939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:32.906328 13939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:32.911787 14076 tablet_service.cc:1467] Processing CreateTablet for tablet 3704d06730f34ea8935fdaf435fb6efc (DEFAULT_TABLE table=test-table [id=7d1a963f0e3c4f85a0287964e08abd37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:32.911979 14216 tablet_service.cc:1467] Processing CreateTablet for tablet 3704d06730f34ea8935fdaf435fb6efc (DEFAULT_TABLE table=test-table [id=7d1a963f0e3c4f85a0287964e08abd37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:32.912017 14356 tablet_service.cc:1467] Processing CreateTablet for tablet 3704d06730f34ea8935fdaf435fb6efc (DEFAULT_TABLE table=test-table [id=7d1a963f0e3c4f85a0287964e08abd37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:32.912097 14076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3704d06730f34ea8935fdaf435fb6efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.912230 14216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3704d06730f34ea8935fdaf435fb6efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.912326 14356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3704d06730f34ea8935fdaf435fb6efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:32.914613 14450 tablet_bootstrap.cc:492] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6: Bootstrap starting.
I20260925 00:43:32.914721 14452 tablet_bootstrap.cc:492] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27: Bootstrap starting.
I20260925 00:43:32.914784 14451 tablet_bootstrap.cc:492] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a: Bootstrap starting.
I20260925 00:43:32.915613 14450 tablet_bootstrap.cc:654] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.915621 14451 tablet_bootstrap.cc:654] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.915617 14452 tablet_bootstrap.cc:654] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:32.916817 14450 tablet_bootstrap.cc:492] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6: No bootstrap required, opened a new log
I20260925 00:43:32.916906 14450 ts_tablet_manager.cc:1397] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 00:43:32.917109 14452 tablet_bootstrap.cc:492] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27: No bootstrap required, opened a new log
I20260925 00:43:32.917201 14452 ts_tablet_manager.cc:1397] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 00:43:32.917354 14450 raft_consensus.cc:348] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.917462 14450 raft_consensus.cc:374] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.917492 14450 raft_consensus.cc:729] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e495c846981446c8c6b099b0a1dd2c6, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.917604 14450 consensus_queue.cc:260] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.917817 14450 ts_tablet_manager.cc:1428] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:32.918169 14452 raft_consensus.cc:348] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.918298 14452 raft_consensus.cc:374] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.918337 14452 raft_consensus.cc:729] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fcad5c64a614a4c9d379852df217d27, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.918467 14452 consensus_queue.cc:260] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.918720 14452 ts_tablet_manager.cc:1428] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:32.920770 14451 tablet_bootstrap.cc:492] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a: No bootstrap required, opened a new log
I20260925 00:43:32.920864 14451 ts_tablet_manager.cc:1397] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260925 00:43:32.921252 14451 raft_consensus.cc:348] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.921367 14451 raft_consensus.cc:374] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:32.921399 14451 raft_consensus.cc:729] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de11d4f98d814ed28e979c8a75e63e2a, State: Initialized, Role: FOLLOWER
I20260925 00:43:32.921506 14451 consensus_queue.cc:260] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:32.921728 14451 ts_tablet_manager.cc:1428] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:33.011445 14510 raft_consensus.cc:482] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:33.011643 14510 raft_consensus.cc:504] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.012288 14510 leader_election.cc:288] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161), 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523)
I20260925 00:43:33.012799 14096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3704d06730f34ea8935fdaf435fb6efc" candidate_uuid: "de11d4f98d814ed28e979c8a75e63e2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e495c846981446c8c6b099b0a1dd2c6" is_pre_election: true
I20260925 00:43:33.012887 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3704d06730f34ea8935fdaf435fb6efc" candidate_uuid: "de11d4f98d814ed28e979c8a75e63e2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fcad5c64a614a4c9d379852df217d27" is_pre_election: true
I20260925 00:43:33.012962 14096 raft_consensus.cc:2454] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de11d4f98d814ed28e979c8a75e63e2a in term 0.
I20260925 00:43:33.013024 14236 raft_consensus.cc:2454] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de11d4f98d814ed28e979c8a75e63e2a in term 0.
I20260925 00:43:33.013303 14309 leader_election.cc:302] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e495c846981446c8c6b099b0a1dd2c6, de11d4f98d814ed28e979c8a75e63e2a; no voters:
I20260925 00:43:33.013520 14510 raft_consensus.cc:2781] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:33.013605 14510 raft_consensus.cc:482] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:33.013652 14510 raft_consensus.cc:3037] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:33.015105 14510 raft_consensus.cc:504] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.015479 14510 leader_election.cc:288] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [CANDIDATE]: Term 1 election: Requested vote from peers 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161), 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523)
I20260925 00:43:33.015853 14096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3704d06730f34ea8935fdaf435fb6efc" candidate_uuid: "de11d4f98d814ed28e979c8a75e63e2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e495c846981446c8c6b099b0a1dd2c6"
I20260925 00:43:33.015947 14236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3704d06730f34ea8935fdaf435fb6efc" candidate_uuid: "de11d4f98d814ed28e979c8a75e63e2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fcad5c64a614a4c9d379852df217d27"
I20260925 00:43:33.016005 14096 raft_consensus.cc:3037] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:33.016091 14236 raft_consensus.cc:3037] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:33.016995 14096 raft_consensus.cc:2454] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de11d4f98d814ed28e979c8a75e63e2a in term 1.
I20260925 00:43:33.017292 14236 raft_consensus.cc:2454] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de11d4f98d814ed28e979c8a75e63e2a in term 1.
I20260925 00:43:33.017308 14309 leader_election.cc:302] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e495c846981446c8c6b099b0a1dd2c6, de11d4f98d814ed28e979c8a75e63e2a; no voters:
I20260925 00:43:33.017524 14510 raft_consensus.cc:2781] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:33.017623 14510 raft_consensus.cc:686] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 1 LEADER]: Becoming Leader. State: Replica: de11d4f98d814ed28e979c8a75e63e2a, State: Running, Role: LEADER
I20260925 00:43:33.018100 14510 consensus_queue.cc:237] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.020325 13939 catalog_manager.cc:5495] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a reported cstate change: term changed from 0 to 1, leader changed from <none> to de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67). New cstate: current_term: 1 leader_uuid: "de11d4f98d814ed28e979c8a75e63e2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } health_report { overall_health: HEALTHY } } }
I20260925 00:43:33.045368 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:33.042124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51968 (local address 127.6.72.67:33151)
0925 00:43:33.042295 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:33.042300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:33.042315 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:43:33.042446 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:33.042449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:33.042501 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:33.042587 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:33.042842 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.043361 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.044262 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.044492 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.044603 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:33.044876 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:33.044957 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:33.045037 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:33.045160 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 00:43:33.045215 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 00:43:33.047814 14236 raft_consensus.cc:1261] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Refusing update from remote peer de11d4f98d814ed28e979c8a75e63e2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:33.047814 14096 raft_consensus.cc:1261] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Refusing update from remote peer de11d4f98d814ed28e979c8a75e63e2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:33.048308 14457 consensus_queue.cc:1035] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:33.048463 14510 consensus_queue.cc:1035] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:33.058822 13937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:33.059003 13937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:33.061242 13937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453: Sending DeleteTablet for 3 replicas of tablet 3704d06730f34ea8935fdaf435fb6efc
I20260925 00:43:33.061858 14076 tablet_service.cc:1514] Processing DeleteTablet for tablet 3704d06730f34ea8935fdaf435fb6efc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:33 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:33988
I20260925 00:43:33.061931 14216 tablet_service.cc:1514] Processing DeleteTablet for tablet 3704d06730f34ea8935fdaf435fb6efc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:33 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:34176
I20260925 00:43:33.062052 14513 tablet_replica.cc:330] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6: stopping tablet replica
I20260925 00:43:33.062064 14356 tablet_service.cc:1514] Processing DeleteTablet for tablet 3704d06730f34ea8935fdaf435fb6efc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:33 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:51942
I20260925 00:43:33.062120 14514 tablet_replica.cc:330] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27: stopping tablet replica
I20260925 00:43:33.062160 14513 raft_consensus.cc:2229] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:33.062235 14512 tablet_replica.cc:330] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a: stopping tablet replica
I20260925 00:43:33.062254 14514 raft_consensus.cc:2229] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:33.062288 14513 raft_consensus.cc:2258] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:33.062345 14512 raft_consensus.cc:2229] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:33.062391 14514 raft_consensus.cc:2258] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:33.062681 14512 raft_consensus.cc:2258] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:33.063148 14513 ts_tablet_manager.cc:1905] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:33.063390 14514 ts_tablet_manager.cc:1905] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:33.063792 13937 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:33.064226 13937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:33.065721 14513 ts_tablet_manager.cc:1918] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:33.065791 14513 log.cc:1196] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/3704d06730f34ea8935fdaf435fb6efc
I20260925 00:43:33.065948 14514 ts_tablet_manager.cc:1918] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:33.066025 14514 log.cc:1196] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/3704d06730f34ea8935fdaf435fb6efc
I20260925 00:43:33.066039 14513 ts_tablet_manager.cc:1939] T 3704d06730f34ea8935fdaf435fb6efc P 8e495c846981446c8c6b099b0a1dd2c6: Deleting consensus metadata
I20260925 00:43:33.066291 14514 ts_tablet_manager.cc:1939] T 3704d06730f34ea8935fdaf435fb6efc P 4fcad5c64a614a4c9d379852df217d27: Deleting consensus metadata
I20260925 00:43:33.066905 13925 catalog_manager.cc:4841] TS 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161): tablet 3704d06730f34ea8935fdaf435fb6efc (table test-table [id=7d1a963f0e3c4f85a0287964e08abd37]) successfully deleted
I20260925 00:43:33.067055 13925 catalog_manager.cc:4841] TS 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523): tablet 3704d06730f34ea8935fdaf435fb6efc (table test-table [id=7d1a963f0e3c4f85a0287964e08abd37]) successfully deleted
I20260925 00:43:33.067715 14512 ts_tablet_manager.cc:1905] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:33.070416 14076 tablet_service.cc:1467] Processing CreateTablet for tablet e30a7b4e53d740899137ffaf5d0f02f5 (DEFAULT_TABLE table=test-table [id=1638d9129b9f4093a8fa75e65b549b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:33.070439 14356 tablet_service.cc:1467] Processing CreateTablet for tablet e30a7b4e53d740899137ffaf5d0f02f5 (DEFAULT_TABLE table=test-table [id=1638d9129b9f4093a8fa75e65b549b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:33.070508 14512 ts_tablet_manager.cc:1918] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:33.070708 14076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30a7b4e53d740899137ffaf5d0f02f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:33.070690 14356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30a7b4e53d740899137ffaf5d0f02f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:33.070787 14512 log.cc:1196] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/3704d06730f34ea8935fdaf435fb6efc
I20260925 00:43:33.071102 14512 ts_tablet_manager.cc:1939] T 3704d06730f34ea8935fdaf435fb6efc P de11d4f98d814ed28e979c8a75e63e2a: Deleting consensus metadata
I20260925 00:43:33.072896 14451 tablet_bootstrap.cc:492] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a: Bootstrap starting.
I20260925 00:43:33.073693 14451 tablet_bootstrap.cc:654] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:33.074328 14450 tablet_bootstrap.cc:492] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6: Bootstrap starting.
I20260925 00:43:33.074692 14216 tablet_service.cc:1467] Processing CreateTablet for tablet e30a7b4e53d740899137ffaf5d0f02f5 (DEFAULT_TABLE table=test-table [id=1638d9129b9f4093a8fa75e65b549b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:33.074968 14216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30a7b4e53d740899137ffaf5d0f02f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:33.075143 14450 tablet_bootstrap.cc:654] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:33.076614 14450 tablet_bootstrap.cc:492] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6: No bootstrap required, opened a new log
I20260925 00:43:33.076593 13923 catalog_manager.cc:4841] TS de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151): tablet 3704d06730f34ea8935fdaf435fb6efc (table test-table [id=7d1a963f0e3c4f85a0287964e08abd37]) successfully deleted
I20260925 00:43:33.076701 14450 ts_tablet_manager.cc:1397] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:33.077137 14450 raft_consensus.cc:348] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.077273 14450 raft_consensus.cc:374] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:33.077337 14450 raft_consensus.cc:729] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e495c846981446c8c6b099b0a1dd2c6, State: Initialized, Role: FOLLOWER
I20260925 00:43:33.077468 14450 consensus_queue.cc:260] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.077618 14452 tablet_bootstrap.cc:492] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27: Bootstrap starting.
I20260925 00:43:33.077723 14451 tablet_bootstrap.cc:492] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a: No bootstrap required, opened a new log
I20260925 00:43:33.077731 14450 ts_tablet_manager.cc:1428] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260925 00:43:33.077801 14451 ts_tablet_manager.cc:1397] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 00:43:33.078179 14451 raft_consensus.cc:348] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.078277 14451 raft_consensus.cc:374] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:33.078307 14451 raft_consensus.cc:729] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de11d4f98d814ed28e979c8a75e63e2a, State: Initialized, Role: FOLLOWER
I20260925 00:43:33.078469 14452 tablet_bootstrap.cc:654] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:33.078419 14451 consensus_queue.cc:260] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.078717 14451 ts_tablet_manager.cc:1428] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 00:43:33.080107 14452 tablet_bootstrap.cc:492] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27: No bootstrap required, opened a new log
I20260925 00:43:33.080199 14452 ts_tablet_manager.cc:1397] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 00:43:33.080665 14452 raft_consensus.cc:348] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.080809 14452 raft_consensus.cc:374] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:33.080852 14452 raft_consensus.cc:729] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fcad5c64a614a4c9d379852df217d27, State: Initialized, Role: FOLLOWER
I20260925 00:43:33.080971 14452 consensus_queue.cc:260] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.081251 14452 ts_tablet_manager.cc:1428] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 00:43:33.385040 14469 raft_consensus.cc:482] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:33.385177 14469 raft_consensus.cc:504] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.385555 14469 leader_election.cc:288] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161), de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151)
I20260925 00:43:33.386060 14096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30a7b4e53d740899137ffaf5d0f02f5" candidate_uuid: "4fcad5c64a614a4c9d379852df217d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e495c846981446c8c6b099b0a1dd2c6" is_pre_election: true
I20260925 00:43:33.386082 14376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30a7b4e53d740899137ffaf5d0f02f5" candidate_uuid: "4fcad5c64a614a4c9d379852df217d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de11d4f98d814ed28e979c8a75e63e2a" is_pre_election: true
I20260925 00:43:33.386238 14376 raft_consensus.cc:2454] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fcad5c64a614a4c9d379852df217d27 in term 0.
I20260925 00:43:33.386258 14096 raft_consensus.cc:2454] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fcad5c64a614a4c9d379852df217d27 in term 0.
I20260925 00:43:33.386669 14167 leader_election.cc:302] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fcad5c64a614a4c9d379852df217d27, de11d4f98d814ed28e979c8a75e63e2a; no voters:
I20260925 00:43:33.386855 14469 raft_consensus.cc:2781] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:33.386915 14469 raft_consensus.cc:482] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:33.386943 14469 raft_consensus.cc:3037] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:33.387940 14469 raft_consensus.cc:504] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.388335 14469 leader_election.cc:288] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [CANDIDATE]: Term 1 election: Requested vote from peers 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161), de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151)
I20260925 00:43:33.388685 14376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30a7b4e53d740899137ffaf5d0f02f5" candidate_uuid: "4fcad5c64a614a4c9d379852df217d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de11d4f98d814ed28e979c8a75e63e2a"
I20260925 00:43:33.388736 14096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30a7b4e53d740899137ffaf5d0f02f5" candidate_uuid: "4fcad5c64a614a4c9d379852df217d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e495c846981446c8c6b099b0a1dd2c6"
I20260925 00:43:33.388821 14376 raft_consensus.cc:3037] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:33.388870 14096 raft_consensus.cc:3037] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:33.389768 14376 raft_consensus.cc:2454] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fcad5c64a614a4c9d379852df217d27 in term 1.
I20260925 00:43:33.390015 14096 raft_consensus.cc:2454] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fcad5c64a614a4c9d379852df217d27 in term 1.
I20260925 00:43:33.390098 14167 leader_election.cc:302] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fcad5c64a614a4c9d379852df217d27, de11d4f98d814ed28e979c8a75e63e2a; no voters:
I20260925 00:43:33.390281 14469 raft_consensus.cc:2781] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:33.390517 14469 raft_consensus.cc:686] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 1 LEADER]: Becoming Leader. State: Replica: 4fcad5c64a614a4c9d379852df217d27, State: Running, Role: LEADER
I20260925 00:43:33.390689 14469 consensus_queue.cc:237] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } }
I20260925 00:43:33.392515 13937 catalog_manager.cc:5495] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66). New cstate: current_term: 1 leader_uuid: "4fcad5c64a614a4c9d379852df217d27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fcad5c64a614a4c9d379852df217d27" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 } health_report { overall_health: UNKNOWN } } }
I20260925 00:43:33.457346 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:33.454097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34196 (local address 127.6.72.66:45523)
0925 00:43:33.454328 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:33.454335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:33.454351 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:33.454396 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:33.454399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:33.454451 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:33.454546 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:33.454808 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.455344 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.456172 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.456506 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.456621 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:33.456871 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:33.456953 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:33.457034 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:33.457127 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 00:43:33.457180 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
I20260925 00:43:33.466820 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:33.464001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:57141 (local address 127.6.72.67:33151)
0925 00:43:33.464197 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:33.464201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:33.464215 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:33.464402 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:33.464405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:33.464446 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:33.464522 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:33.464713 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.465204 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.465970 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.466188 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.466261 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:33.466485 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:33.466544 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:33.466597 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:33.466675 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 00:43:33.466715 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260925 00:43:33.466848 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:33.463872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33151 (local address 127.6.72.66:57141)
0925 00:43:33.464151 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:33.464163 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:33.464282 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:33.464540 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:33.464542 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:33.464548 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:43:33.464624 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:33.464628 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.465291 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.465293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:33.465876 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:33.465883 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.466280 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.466282 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:33.466339 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:33.466377 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:43:33.466617 (+ 240us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:43:33.466622 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 00:43:33.466669 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 00:43:33.466739 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
I20260925 00:43:33.473107 14096 raft_consensus.cc:1261] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Refusing update from remote peer 4fcad5c64a614a4c9d379852df217d27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:33.473119 14376 raft_consensus.cc:1261] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Refusing update from remote peer 4fcad5c64a614a4c9d379852df217d27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:33.473475 14469 consensus_queue.cc:1035] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e495c846981446c8c6b099b0a1dd2c6" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:33.473624 14538 consensus_queue.cc:1035] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de11d4f98d814ed28e979c8a75e63e2a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:33.490366 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:33.486681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:42643 (local address 127.6.72.66:45523)
0925 00:43:33.486798 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:33.486802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:33.486972 (+ 170us) server_negotiation.cc:407] Connection header received
0925 00:43:33.487114 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:33.487120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:33.487177 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:33.487258 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:33.487513 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.488085 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.489101 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.489399 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.489506 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:33.489843 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:33.489941 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:33.490009 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:33.490162 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 00:43:33.490229 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 00:43:33.490391 14548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:33.486582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:45523 (local address 127.6.72.67:42643)
0925 00:43:33.486878 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:33.486892 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:33.486989 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:33.487284 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:33.487290 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:33.487300 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 00:43:33.487417 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:33.487424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.488175 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.488177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:33.488888 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:33.488898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:33.489511 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:33.489514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:33.489620 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:33.489666 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 00:43:33.490029 (+ 363us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 00:43:33.490035 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 00:43:33.490106 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 00:43:33.490198 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":55,"threads_started":1}
I20260925 00:43:33.509516 13937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:33.509686 13937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:33.511600 13937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d84b0aafaaec4bd792811e7cc3821453: Sending DeleteTablet for 3 replicas of tablet e30a7b4e53d740899137ffaf5d0f02f5
I20260925 00:43:33.512149 14076 tablet_service.cc:1514] Processing DeleteTablet for tablet e30a7b4e53d740899137ffaf5d0f02f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:33 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:33988
I20260925 00:43:33.512159 14216 tablet_service.cc:1514] Processing DeleteTablet for tablet e30a7b4e53d740899137ffaf5d0f02f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:33 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:34176
I20260925 00:43:33.512290 14356 tablet_service.cc:1514] Processing DeleteTablet for tablet e30a7b4e53d740899137ffaf5d0f02f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:33 UTC) from {username='oryx', principal='oryx/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:51942
I20260925 00:43:33.512389 14514 tablet_replica.cc:330] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27: stopping tablet replica
I20260925 00:43:33.512501 14512 tablet_replica.cc:330] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a: stopping tablet replica
I20260925 00:43:33.512598 14514 raft_consensus.cc:2229] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:33.512636 14512 raft_consensus.cc:2229] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:33.512780 14512 raft_consensus.cc:2258] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:33.512790 14514 raft_consensus.cc:2258] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:33.512974 14513 tablet_replica.cc:330] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6: stopping tablet replica
I20260925 00:43:33.513144 14513 raft_consensus.cc:2229] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:33.513293 14513 raft_consensus.cc:2258] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:33.513463 14512 ts_tablet_manager.cc:1905] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:33.513995 14513 ts_tablet_manager.cc:1905] T e30a7b4e53d740899137ffaf5d0f02f5 P 8e495c846981446c8c6b099b0a1dd2c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:33.514424 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 14011
I20260925 00:43:33.515888 14514 ts_tablet_manager.cc:1905] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:33.517937 14514 ts_tablet_manager.cc:1918] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:33.518034 14514 log.cc:1196] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/e30a7b4e53d740899137ffaf5d0f02f5
I20260925 00:43:33.518280 14514 ts_tablet_manager.cc:1939] T e30a7b4e53d740899137ffaf5d0f02f5 P 4fcad5c64a614a4c9d379852df217d27: Deleting consensus metadata
I20260925 00:43:33.519052 13925 catalog_manager.cc:4841] TS 4fcad5c64a614a4c9d379852df217d27 (127.6.72.66:45523): tablet e30a7b4e53d740899137ffaf5d0f02f5 (table test-table [id=1638d9129b9f4093a8fa75e65b549b84]) successfully deleted
I20260925 00:43:33.521159 14512 ts_tablet_manager.cc:1918] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 00:43:33.521245 14512 log.cc:1196] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/e30a7b4e53d740899137ffaf5d0f02f5
I20260925 00:43:33.521574 14512 ts_tablet_manager.cc:1939] T e30a7b4e53d740899137ffaf5d0f02f5 P de11d4f98d814ed28e979c8a75e63e2a: Deleting consensus metadata
W20260925 00:43:33.522368 13925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:35161 (error 108)
I20260925 00:43:33.522558 13923 catalog_manager.cc:4841] TS de11d4f98d814ed28e979c8a75e63e2a (127.6.72.67:33151): tablet e30a7b4e53d740899137ffaf5d0f02f5 (table test-table [id=1638d9129b9f4093a8fa75e65b549b84]) successfully deleted
I20260925 00:43:33.523157 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 14151
I20260925 00:43:33.523571 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:33.523122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35161 (local address 127.0.0.1:33998)
0925 00:43:33.523358 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:33.523438 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:35161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
W20260925 00:43:33.523975 13925 catalog_manager.cc:4584] TS 8e495c846981446c8c6b099b0a1dd2c6 (127.6.72.65:35161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e30a7b4e53d740899137ffaf5d0f02f5: Network error: Client connection negotiation failed: client connection to 127.6.72.65:35161: connect: Connection refused (error 111)
I20260925 00:43:33.530575 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 14291
I20260925 00:43:33.537703 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 13907
2026-09-25T00:43:33Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3608 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:33 dist-test-slave-w8nh krb5kdc[14554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:33 dist-test-slave-w8nh krb5kdc[14554](info): set up 2 sockets
Sep 25 00:43:33 dist-test-slave-w8nh krb5kdc[14554](info): commencing operation
krb5kdc: starting...
W20260925 00:43:34.597184 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:34 dist-test-slave-w8nh krb5kdc[14554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:43:34.720507 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:45277
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:45277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:34.832324 14567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:34.832674 14567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:34.832749 14567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:34.836858 14567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:34.836966 14567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:34.837007 14567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:34.837046 14567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:34.837086 14567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:34.842540 14567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:45277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:45277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:34.843744 14567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:34.844831 14567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 00:43:34.853552 14567 server_base.cc:1034] running on GCE node
W20260925 00:43:34.853499 14573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:34.853497 14572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:34.853494 14575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:34.854321 14567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:34.855355 14567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:34.856508 14567 hybrid_clock.cc:648] HybridClock initialized: now 1790297014856491 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T00:43:34Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1330 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:34 dist-test-slave-w8nh krb5kdc[14580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:34 dist-test-slave-w8nh krb5kdc[14580](info): set up 2 sockets
Sep 25 00:43:34 dist-test-slave-w8nh krb5kdc[14580](info): commencing operation
krb5kdc: starting...
W20260925 00:43:35.929231 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:36 dist-test-slave-w8nh krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:43:36.055970 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:35957
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:35957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:36.171375 14593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:36.171659 14593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:36.171701 14593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:36.175772 14593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:36.175848 14593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:36.175868 14593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:36.175884 14593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:36.175900 14593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:36.181370 14593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:35957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:35957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:36.182554 14593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:36.183710 14593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:36.192793 14598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:36.192873 14601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:36.192883 14593 server_base.cc:1034] running on GCE node
W20260925 00:43:36.192822 14599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:36.193712 14593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:36.194900 14593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:36.196126 14593 hybrid_clock.cc:648] HybridClock initialized: now 1790297016196092 us; error 49 us; skew 500 ppm
Sep 25 00:43:36 dist-test-slave-w8nh krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297016, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:36.199144 14593 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:43:36.200114 14593 webserver.cc:460] Webserver started at http://127.6.72.126:39737/ using document root <none> and password file <none>
I20260925 00:43:36.200766 14593 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:36.200860 14593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:36.201100 14593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:36.203197 14593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "4ab8de88b9a341a2bb0a275f5f0964f1"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "52418376064fdbf526fc90333925dded"
server_key_iv: "cb93e321a52e6f65f5ef72c2210031bf"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.203763 14593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "4ab8de88b9a341a2bb0a275f5f0964f1"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "52418376064fdbf526fc90333925dded"
server_key_iv: "cb93e321a52e6f65f5ef72c2210031bf"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.207710 14593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:43:36.210445 14608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:36.211838 14593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:36.212013 14593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "4ab8de88b9a341a2bb0a275f5f0964f1"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "52418376064fdbf526fc90333925dded"
server_key_iv: "cb93e321a52e6f65f5ef72c2210031bf"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.212188 14593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:36.234488 14593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:36.238229 14593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:36.238461 14593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:36.246554 14593 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:35957
I20260925 00:43:36.246590 14660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:35957 every 8 connection(s)
I20260925 00:43:36.247618 14593 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:36.250454 14661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:36.253166 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 14593
I20260925 00:43:36.253312 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:36.253605 6433 external_mini_cluster.cc:1428] Setting key 786ba95c2c65f1df0cd6ba19130ff7c7
I20260925 00:43:36.256881 14661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1: Bootstrap starting.
I20260925 00:43:36.259078 14661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:36.259807 14661 log.cc:824] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:36.261686 14661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1: No bootstrap required, opened a new log
Sep 25 00:43:36 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:36.264618 14661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } }
I20260925 00:43:36.264865 14661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:36.264943 14661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ab8de88b9a341a2bb0a275f5f0964f1, State: Initialized, Role: FOLLOWER
I20260925 00:43:36.265376 14661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } }
I20260925 00:43:36.265532 14661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:36.265657 14661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:36.265779 14661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:36.266827 14661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } }
I20260925 00:43:36.267199 14661 leader_election.cc:302] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ab8de88b9a341a2bb0a275f5f0964f1; no voters:
I20260925 00:43:36.267549 14661 leader_election.cc:288] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:36.267611 14666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:36.267810 14666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 1 LEADER]: Becoming Leader. State: Replica: 4ab8de88b9a341a2bb0a275f5f0964f1, State: Running, Role: LEADER
I20260925 00:43:36.268159 14666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } }
I20260925 00:43:36.268672 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:36.255167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60930 (local address 127.6.72.126:35957)
0925 00:43:36.255615 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:36.255632 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:36.255669 (+ 37us) server_negotiation.cc:407] Connection header received
0925 00:43:36.256330 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:36.256349 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:36.256693 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:36.257036 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:36.257838 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:36.258824 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:36.259748 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:36.260337 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:36.260504 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:36.262991 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:36.262997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:36.263009 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:36.263035 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:36.265502 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:36.266036 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:36.266041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:36.266045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:36.266178 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:36.266463 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:36.266465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:36.266466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:36.266834 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:36.267029 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:36.267292 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 00:43:36.267674 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
I20260925 00:43:36.268833 14661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:36.269757 14667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } } }
I20260925 00:43:36.269886 14667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:36.269992 14668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ab8de88b9a341a2bb0a275f5f0964f1. Latest consensus state: current_term: 1 leader_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } } }
I20260925 00:43:36.270085 14668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:36.270886 14673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:36.273983 14673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:36.279124 14673 catalog_manager.cc:1347] Generated new cluster ID: 83e30718a0394b3ba5564fd16ec0a015
I20260925 00:43:36.279199 14673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:36.318233 14673 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:36.319299 14673 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:36.325757 14673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1: Generated new TSK 0
I20260925 00:43:36.326551 14673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:43:36.360358 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35957
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:36.478961 14687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:36.479331 14687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:36.479423 14687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:36.483806 14687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:36.483894 14687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:36.483990 14687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:36.489771 14687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:36.490927 14687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:36.491997 14687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:36.502218 14693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:36.502204 14692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:36.502245 14687 server_base.cc:1034] running on GCE node
W20260925 00:43:36.502194 14695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:36.502863 14687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:36.503839 14687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:36.505234 14687 hybrid_clock.cc:648] HybridClock initialized: now 1790297016505192 us; error 130 us; skew 500 ppm
Sep 25 00:43:36 dist-test-slave-w8nh krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297016, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:36.508381 14687 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:43:36.509375 14687 webserver.cc:460] Webserver started at http://127.6.72.65:46363/ using document root <none> and password file <none>
I20260925 00:43:36.509960 14687 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:36.510059 14687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:36.510290 14687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:36.512481 14687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "6ae09f4e83f140eaaf95d5d7309fd321"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "d976bc52ef10703744eb1af0ad6a2937"
server_key_iv: "bd9075301597dc0700950844e04b41df"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.513072 14687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "6ae09f4e83f140eaaf95d5d7309fd321"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "d976bc52ef10703744eb1af0ad6a2937"
server_key_iv: "bd9075301597dc0700950844e04b41df"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.517208 14687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:43:36.519809 14702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:36.521225 14687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:36.521368 14687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "6ae09f4e83f140eaaf95d5d7309fd321"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "d976bc52ef10703744eb1af0ad6a2937"
server_key_iv: "bd9075301597dc0700950844e04b41df"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.521504 14687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:36.532297 14687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:36.535655 14687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:36.535866 14687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:36.536562 14687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:36.537415 14687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:36.537485 14687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:36.537554 14687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:36.537601 14687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:36.546732 14687 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:33371
I20260925 00:43:36.546767 14815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:33371 every 8 connection(s)
I20260925 00:43:36.547698 14687 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
Sep 25 00:43:36 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297016, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:36.557880 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 14687
I20260925 00:43:36.558022 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:36.558353 6433 external_mini_cluster.cc:1428] Setting key f35c9678c53a5a1d6ec130da8740031d
I20260925 00:43:36.562755 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:36.549727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:43553 (local address 127.6.72.126:35957)
0925 00:43:36.549889 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:36.549893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:36.550738 (+ 845us) server_negotiation.cc:407] Connection header received
0925 00:43:36.551977 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:36.551981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:36.552067 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:36.552144 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:36.553362 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:36.553917 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:36.555081 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:36.555340 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:36.555435 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:36.558226 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:36.558230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:36.558233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:36.558261 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:36.560098 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:36.560794 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:36.560797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:36.560798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:36.560879 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:36.561344 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:36.561347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:36.561348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:36.561532 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:36.561616 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:36.562332 (+ 716us) server_negotiation.cc:299] Negotiation successful
0925 00:43:36.562520 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 00:43:36.563577 14818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:36.550100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:43553)
0925 00:43:36.550559 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:36.550623 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:36.551764 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:36.552334 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:36.552341 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:36.552729 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:36.553213 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:36.553226 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:36.554035 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:36.554038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:36.554954 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:36.554964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:36.555489 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:36.555492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:36.555655 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:36.556377 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:36.556402 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:36.558023 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:36.560234 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:36.560265 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:36.560284 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:36.560667 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:36.560973 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:36.560979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:36.560984 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:36.561231 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:36.561633 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:36.561642 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:36.562035 (+ 393us) client_negotiation.cc:769] Sending connection context
0925 00:43:36.562228 (+ 193us) client_negotiation.cc:240] Negotiation successful
0925 00:43:36.562531 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260925 00:43:36.564970 14816 heartbeater.cc:347] Connected to a master server at 127.6.72.126:35957
I20260925 00:43:36.565258 14816 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:36.565765 14816 heartbeater.cc:511] Master 127.6.72.126:35957 requested a full tablet report, sending...
I20260925 00:43:36.567338 14625 ts_manager.cc:194] Registered new tserver with Master: 6ae09f4e83f140eaaf95d5d7309fd321 (127.6.72.65:33371)
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:43:36.568717 14625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:43553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:43:36.588527 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35957
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:36.704450 14821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:36.704771 14821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:36.704852 14821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:36.709466 14821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:36.709583 14821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:36.709692 14821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:36.716184 14821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:36.717551 14821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:36.718771 14821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 00:43:36.729475 14821 server_base.cc:1034] running on GCE node
W20260925 00:43:36.729406 14827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:36.729406 14829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:36.729401 14826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:36.730057 14821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:36.730806 14821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:36.732007 14821 hybrid_clock.cc:648] HybridClock initialized: now 1790297016731986 us; error 33 us; skew 500 ppm
Sep 25 00:43:36 dist-test-slave-w8nh krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297016, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:36.734833 14821 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:43:36.735772 14821 webserver.cc:460] Webserver started at http://127.6.72.66:39529/ using document root <none> and password file <none>
I20260925 00:43:36.736357 14821 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:36.736423 14821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:36.736599 14821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:36.738583 14821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "54dac4a3b1734708ac7d062050a482f9"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "fd55186bf543a6b6d74f51f3ae6ecb7c"
server_key_iv: "a5e1baaeb08ac55286566bf074420a36"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.739127 14821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "54dac4a3b1734708ac7d062050a482f9"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "fd55186bf543a6b6d74f51f3ae6ecb7c"
server_key_iv: "a5e1baaeb08ac55286566bf074420a36"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.743026 14821 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:43:36.745569 14836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:36.746925 14821 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:36.747117 14821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "54dac4a3b1734708ac7d062050a482f9"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "fd55186bf543a6b6d74f51f3ae6ecb7c"
server_key_iv: "a5e1baaeb08ac55286566bf074420a36"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.747243 14821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:36.761313 14821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:36.764613 14821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:36.764819 14821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:36.765445 14821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:36.766290 14821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:36.766392 14821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:36.766434 14821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:36.766449 14821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:36.776175 14821 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:44619
I20260925 00:43:36.776191 14949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:44619 every 8 connection(s)
I20260925 00:43:36.777123 14821 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:36.785786 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 14821
I20260925 00:43:36.785943 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:36.786274 6433 external_mini_cluster.cc:1428] Setting key d77f3241df698c9cfd657bd98444e156
Sep 25 00:43:36 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297016, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:36.791700 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:36.778921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:38035 (local address 127.6.72.126:35957)
0925 00:43:36.779086 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:36.779090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:36.779929 (+ 839us) server_negotiation.cc:407] Connection header received
0925 00:43:36.781262 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:36.781265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:36.781315 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:36.781396 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:36.782591 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:36.783120 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:36.784499 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:36.784726 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:36.784815 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:36.787622 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:36.787627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:36.787629 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:36.787659 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:36.789336 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:36.789960 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:36.789963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:36.789965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:36.790040 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:36.790463 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:36.790466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:36.790468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:36.790681 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:36.790813 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:36.791432 (+ 619us) server_negotiation.cc:299] Negotiation successful
0925 00:43:36.791571 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 00:43:36.793011 14952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:36.779308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:38035)
0925 00:43:36.779769 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:36.779817 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:36.781028 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:36.781575 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:36.781582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:36.781947 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:36.782427 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:36.782439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:36.783254 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:36.783257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:36.784360 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:36.784371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:36.784871 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:36.784875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:36.785035 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:36.785640 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:36.785666 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:36.787374 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:36.789456 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:36.789463 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:36.789484 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:36.789856 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:36.790134 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:36.790137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:36.790138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:36.790333 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:36.790807 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:36.790817 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:36.791160 (+ 343us) client_negotiation.cc:769] Sending connection context
0925 00:43:36.791375 (+ 215us) client_negotiation.cc:240] Negotiation successful
0925 00:43:36.791773 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":104,"threads_started":1}
I20260925 00:43:36.794414 14950 heartbeater.cc:347] Connected to a master server at 127.6.72.126:35957
I20260925 00:43:36.794683 14950 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:36.795192 14950 heartbeater.cc:511] Master 127.6.72.126:35957 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:43:36.796329 14625 ts_manager.cc:194] Registered new tserver with Master: 54dac4a3b1734708ac7d062050a482f9 (127.6.72.66:44619)
I20260925 00:43:36.796898 14625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:38035
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:43:36.815374 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35957
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:36.932399 14955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:36.932684 14955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:36.932770 14955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:36.937001 14955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:36.937086 14955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:36.937191 14955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:36.942878 14955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:35957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:36.944067 14955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:36.945181 14955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:36.955538 14961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:36.955686 14963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:36.955538 14960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:36.955839 14955 server_base.cc:1034] running on GCE node
I20260925 00:43:36.956385 14955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:36.957129 14955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:36.958292 14955 hybrid_clock.cc:648] HybridClock initialized: now 1790297016958257 us; error 50 us; skew 500 ppm
Sep 25 00:43:36 dist-test-slave-w8nh krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297016, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:36.961336 14955 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:43:36.962347 14955 webserver.cc:460] Webserver started at http://127.6.72.67:45731/ using document root <none> and password file <none>
I20260925 00:43:36.962939 14955 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:36.962999 14955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:36.963228 14955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:36.965394 14955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "d1a3f35c1ab140f482cf3de90ac8cfa4"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "b12868d2d58ce08d7cf0bcb4a78ec04d"
server_key_iv: "fc7ed810cf1569a1c004d144a42fd4f9"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.965955 14955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "d1a3f35c1ab140f482cf3de90ac8cfa4"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "b12868d2d58ce08d7cf0bcb4a78ec04d"
server_key_iv: "fc7ed810cf1569a1c004d144a42fd4f9"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.969888 14955 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:43:36.972606 14970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:36.973824 14955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:36.973960 14955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "d1a3f35c1ab140f482cf3de90ac8cfa4"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "b12868d2d58ce08d7cf0bcb4a78ec04d"
server_key_iv: "fc7ed810cf1569a1c004d144a42fd4f9"
server_key_version: "encryptionkey@0"
I20260925 00:43:36.974074 14955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:36.999701 14955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:37.002961 14955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:37.003175 14955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:37.003821 14955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:37.004714 14955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:37.004758 14955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.004824 14955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:37.004859 14955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.013993 14955 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:44679
I20260925 00:43:37.014010 15083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:44679 every 8 connection(s)
I20260925 00:43:37.014947 14955 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:37.022776 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 14955
I20260925 00:43:37.022919 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:37.023247 6433 external_mini_cluster.cc:1428] Setting key 9b0242f8ffa6caa756da969e8da4ea67
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297016, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:37.029083 14664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.016888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:44169 (local address 127.6.72.126:35957)
0925 00:43:37.017044 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.017049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.017976 (+ 927us) server_negotiation.cc:407] Connection header received
0925 00:43:37.019234 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.019237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.019284 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.019358 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.020599 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.021131 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.022220 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.022423 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.022500 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.025321 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:37.025325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:37.025327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:37.025350 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:37.026721 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:37.027231 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:37.027234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:37.027236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:37.027321 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:37.027732 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:37.027735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:37.027737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:37.028008 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:37.028108 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:37.028788 (+ 680us) server_negotiation.cc:299] Negotiation successful
0925 00:43:37.028918 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 00:43:37.029963 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.017365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:44169)
0925 00:43:37.017815 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:37.017860 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:37.019008 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:37.019533 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:37.019540 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:37.019905 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:37.020469 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.020484 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.021235 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.021237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.022110 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.022119 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.022575 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.022580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.022809 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.023518 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:37.023540 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:37.025137 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:37.026839 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:37.026845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:37.026863 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:37.027123 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:37.027443 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:37.027446 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:37.027448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:37.027620 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:37.028115 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:37.028125 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:37.028510 (+ 385us) client_negotiation.cc:769] Sending connection context
0925 00:43:37.028749 (+ 239us) client_negotiation.cc:240] Negotiation successful
0925 00:43:37.029114 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":103,"threads_started":1}
I20260925 00:43:37.031096 15084 heartbeater.cc:347] Connected to a master server at 127.6.72.126:35957
I20260925 00:43:37.031349 15084 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:37.031862 15084 heartbeater.cc:511] Master 127.6.72.126:35957 requested a full tablet report, sending...
I20260925 00:43:37.032975 14625 ts_manager.cc:194] Registered new tserver with Master: d1a3f35c1ab140f482cf3de90ac8cfa4 (127.6.72.67:44679)
I20260925 00:43:37.033555 14625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:44169
I20260925 00:43:37.037297 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab.
Entry for principal oryx/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab.
WARNING: no policy specified for oryx/127.6.72.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab.
Entry for principal oryx/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab.
WARNING: no policy specified for oryx/127.6.72.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.72.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab.
Entry for principal oryx/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab.
I20260925 00:43:37.119088 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 14687
I20260925 00:43:37.124840 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 14821
I20260925 00:43:37.133226 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 14955
I20260925 00:43:37.138685 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 14593
I20260925 00:43:37.145138 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:35957
--webserver_interface=127.6.72.126
--webserver_port=39737
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:35957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:37.260381 15093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:37.260749 15093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:37.260838 15093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:37.265143 15093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:37.265251 15093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:37.265286 15093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:37.265329 15093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:37.265370 15093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:37.271019 15093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:35957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:35957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:37.272332 15093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:37.273404 15093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:37.282433 15099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:37.282472 15098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:37.282577 15093 server_base.cc:1034] running on GCE node
W20260925 00:43:37.282436 15101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:37.283336 15093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:37.284672 15093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:37.285840 15093 hybrid_clock.cc:648] HybridClock initialized: now 1790297017285810 us; error 42 us; skew 500 ppm
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:37.288810 15093 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:43:37.289830 15093 webserver.cc:460] Webserver started at http://127.6.72.126:39737/ using document root <none> and password file <none>
I20260925 00:43:37.290382 15093 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:37.290457 15093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:37.294595 15093 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:43:37.297043 15108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.298404 15093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:37.298557 15093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "4ab8de88b9a341a2bb0a275f5f0964f1"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "52418376064fdbf526fc90333925dded"
server_key_iv: "cb93e321a52e6f65f5ef72c2210031bf"
server_key_version: "encryptionkey@0"
I20260925 00:43:37.299139 15093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:37.312620 15093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:37.316186 15093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:37.316470 15093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:37.324172 15093 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:35957
I20260925 00:43:37.324219 15160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:35957 every 8 connection(s)
I20260925 00:43:37.325356 15093 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:37.330720 15161 sys_catalog.cc:263] Verifying existing consensus state
I20260925 00:43:37.332170 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 15093
I20260925 00:43:37.332773 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:33371
--local_ip_for_outbound_sockets=127.6.72.65
--tserver_master_addrs=127.6.72.126:35957
--webserver_port=46363
--webserver_interface=127.6.72.65
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.72.65
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:43:37.333102 15161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1: Bootstrap starting.
I20260925 00:43:37.351171 15161 log.cc:824] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:37.359220 15161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:43:37.359656 15161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1: Bootstrap complete.
I20260925 00:43:37.364079 15161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } }
I20260925 00:43:37.364622 15161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ab8de88b9a341a2bb0a275f5f0964f1, State: Initialized, Role: FOLLOWER
I20260925 00:43:37.365191 15161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } }
I20260925 00:43:37.365312 15161 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:37.365408 15161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:37.365510 15161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 1 FOLLOWER]: Advancing to term 2
I20260925 00:43:37.367774 15161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } }
I20260925 00:43:37.368156 15161 leader_election.cc:302] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ab8de88b9a341a2bb0a275f5f0964f1; no voters:
I20260925 00:43:37.368589 15161 leader_election.cc:288] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 00:43:37.368685 15165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 00:43:37.368925 15165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [term 2 LEADER]: Becoming Leader. State: Replica: 4ab8de88b9a341a2bb0a275f5f0964f1, State: Running, Role: LEADER
I20260925 00:43:37.369299 15165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } }
I20260925 00:43:37.369928 15161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:37.370857 15167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ab8de88b9a341a2bb0a275f5f0964f1. Latest consensus state: current_term: 2 leader_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } } }
I20260925 00:43:37.370992 15167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:37.371248 15166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8de88b9a341a2bb0a275f5f0964f1" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 35957 } } }
I20260925 00:43:37.371320 15166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:37.371376 15174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:37.374863 15174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:37.375597 15174 catalog_manager.cc:1259] Loaded cluster ID: 83e30718a0394b3ba5564fd16ec0a015
I20260925 00:43:37.375698 15174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:37.378290 15174 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:37.379025 15174 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ab8de88b9a341a2bb0a275f5f0964f1: Loaded TSK: 0
I20260925 00:43:37.380421 15174 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 00:43:37.452797 15163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:37.453088 15163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:37.453133 15163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:37.457414 15163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:37.457496 15163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:37.457561 15163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:37.463152 15163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.65.keytab
--principal=oryx/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:33371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=46363
--tserver_master_addrs=127.6.72.126:35957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:37.464341 15163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:37.465402 15163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:37.475478 15187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:37.475455 15190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:37.475468 15188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:37.475782 15163 server_base.cc:1034] running on GCE node
I20260925 00:43:37.476230 15163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:37.476955 15163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:37.478168 15163 hybrid_clock.cc:648] HybridClock initialized: now 1790297017478140 us; error 44 us; skew 500 ppm
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297017, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:37.481776 15163 init.cc:377] Logged in from keytab as oryx/127.6.72.65@KRBTEST.COM (short username oryx)
I20260925 00:43:37.482961 15163 webserver.cc:460] Webserver started at http://127.6.72.65:46363/ using document root <none> and password file <none>
I20260925 00:43:37.483714 15163 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:37.483826 15163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:37.489245 15163 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:37.492158 15197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.493856 15163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:37.494004 15163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "6ae09f4e83f140eaaf95d5d7309fd321"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "d976bc52ef10703744eb1af0ad6a2937"
server_key_iv: "bd9075301597dc0700950844e04b41df"
server_key_version: "encryptionkey@0"
I20260925 00:43:37.494772 15163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:37.506843 15163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:37.510440 15163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:37.510653 15163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:37.511267 15163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:37.512151 15163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:37.512197 15163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.512293 15163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:37.512327 15163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.521360 15163 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:33371
I20260925 00:43:37.521404 15310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:33371 every 8 connection(s)
I20260925 00:43:37.522460 15163 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:37.530408 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 15163
I20260925 00:43:37.530956 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:44619
--local_ip_for_outbound_sockets=127.6.72.66
--tserver_master_addrs=127.6.72.126:35957
--webserver_port=39529
--webserver_interface=127.6.72.66
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.72.66
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:37.537155 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.524940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:59163)
0925 00:43:37.525416 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:37.525469 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:37.526694 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:37.527812 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:37.527819 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:37.528349 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:37.528926 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.528940 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.530266 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.530271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.531714 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.531729 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.532732 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.532739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.532966 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.533683 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:37.533708 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:37.536318 (+ 2610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":132,"threads_started":1}
W20260925 00:43:37.537726 15313 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:37.524741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:59163 (local address 127.6.72.126:35957)
0925 00:43:37.525233 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.525256 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.525606 (+ 350us) server_negotiation.cc:407] Connection header received
0925 00:43:37.527096 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.527108 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.527431 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.527725 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.529096 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.530092 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.531870 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.532533 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.532684 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.536664 (+ 3980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:59163: BlockingRecv error: recv got EOF from 127.6.72.65:59163 (error 108)
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":120,"threads_started":1}
W20260925 00:43:37.537825 15311 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:35957 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.72.126:35957: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:37.543936 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.538386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:38983)
0925 00:43:37.538513 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:37.538529 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:37.538636 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:37.538994 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:37.538997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:37.539179 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:37.539271 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.539277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.540060 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.540063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.540692 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.540701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.541192 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.541195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.541285 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.541822 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:37.541839 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:37.543787 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 00:43:37.544414 15316 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:37.538450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:38983 (local address 127.6.72.126:35957)
0925 00:43:37.538786 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.538792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.538804 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:43:37.538840 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.538842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.538879 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.538977 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.539390 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.539939 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.540827 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.541075 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.541190 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.543912 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:38983: BlockingRecv error: recv got EOF from 127.6.72.65:38983 (error 108)
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":135,"threads_started":1}
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:37.550246 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.544562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:51145)
0925 00:43:37.544749 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:37.544764 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:37.544843 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:37.545142 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:37.545145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:37.545270 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:37.545342 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.545346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.546386 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.546389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.546899 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.546906 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.547487 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.547490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.547554 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.548099 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:37.548111 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:37.550136 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124}
W20260925 00:43:37.550429 15316 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:37.544764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:51145 (local address 127.6.72.126:35957)
0925 00:43:37.544899 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.544904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.544918 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:37.544959 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.544961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.545008 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.545097 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.545466 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.546274 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.547007 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.547353 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.547455 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.550312 (+ 2857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:51145: BlockingRecv error: recv got EOF from 127.6.72.65:51145 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260925 00:43:37.550580 15311 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 00:43:37.649384 15315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:37.649742 15315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:37.649825 15315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:37.654052 15315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:37.654170 15315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:37.654274 15315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:37.659986 15315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.66.keytab
--principal=oryx/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:44619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=39529
--tserver_master_addrs=127.6.72.126:35957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:37.661249 15315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:37.662323 15315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:37.672569 15321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:37.672775 15324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:37.672580 15322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:37.672804 15315 server_base.cc:1034] running on GCE node
I20260925 00:43:37.673422 15315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:37.674059 15315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:37.675266 15315 hybrid_clock.cc:648] HybridClock initialized: now 1790297017675251 us; error 31 us; skew 500 ppm
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297017, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:37.678359 15315 init.cc:377] Logged in from keytab as oryx/127.6.72.66@KRBTEST.COM (short username oryx)
I20260925 00:43:37.679371 15315 webserver.cc:460] Webserver started at http://127.6.72.66:39529/ using document root <none> and password file <none>
I20260925 00:43:37.679942 15315 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:37.680023 15315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:37.684140 15315 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:37.686771 15331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.688372 15315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:37.688516 15315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "54dac4a3b1734708ac7d062050a482f9"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "fd55186bf543a6b6d74f51f3ae6ecb7c"
server_key_iv: "a5e1baaeb08ac55286566bf074420a36"
server_key_version: "encryptionkey@0"
I20260925 00:43:37.689127 15315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:37.700882 15315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:37.704514 15315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:37.704728 15315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:37.705348 15315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:37.706226 15315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:37.706300 15315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.706368 15315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:37.706418 15315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.715534 15315 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:44619
I20260925 00:43:37.715552 15444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:44619 every 8 connection(s)
I20260925 00:43:37.716652 15315 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:37.719138 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 15315
I20260925 00:43:37.719658 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:44679
--local_ip_for_outbound_sockets=127.6.72.67
--tserver_master_addrs=127.6.72.126:35957
--webserver_port=45731
--webserver_interface=127.6.72.67
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.72.67
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 00:43:37.730418 15316 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:37.718489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:39063 (local address 127.6.72.126:35957)
0925 00:43:37.718630 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.718634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.719538 (+ 904us) server_negotiation.cc:407] Connection header received
0925 00:43:37.721562 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.721567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.721642 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.721757 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.723283 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.723811 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.725856 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.726083 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.726163 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.730273 (+ 4110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:39063: BlockingRecv error: recv got EOF from 127.6.72.66:39063 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:43:37.731099 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.718825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:39063)
0925 00:43:37.719345 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:37.719406 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:37.721276 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:37.721962 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:37.721969 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:37.722353 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:37.723110 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.723134 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.723984 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.723989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.725703 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.725717 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.726259 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.726266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.726520 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.727227 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:37.727256 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:37.730140 (+ 2884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":110,"threads_started":1}
W20260925 00:43:37.731748 15445 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:35957 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.72.126:35957: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:37.737360 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.732225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:47995)
0925 00:43:37.732359 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:37.732373 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:37.732460 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:37.732709 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:37.732712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:37.732886 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:37.732985 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.732990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.733725 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.733731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.734300 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.734309 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.734752 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.734755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.734846 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.735394 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:37.735406 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:37.737207 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260925 00:43:37.737439 15316 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:37.732352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:47995 (local address 127.6.72.126:35957)
0925 00:43:37.732487 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.732494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.732510 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:37.732560 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.732563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.732602 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.732704 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.733100 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.733607 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.734419 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.734636 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.734750 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.737323 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:47995: BlockingRecv error: recv got EOF from 127.6.72.66:47995 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:37.743211 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.738005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:55733)
0925 00:43:37.738100 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:37.738114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:37.738192 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:37.738424 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:37.738426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:37.738548 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:37.738617 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.738621 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.739644 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.739646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.740143 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.740150 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.740744 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.740747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.740804 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.741298 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:37.741309 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:37.743075 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 00:43:37.743439 15316 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:37.738081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:55733 (local address 127.6.72.126:35957)
0925 00:43:37.738198 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.738203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.738245 (+ 42us) server_negotiation.cc:407] Connection header received
0925 00:43:37.738285 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.738288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.738329 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.738404 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.738716 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.739523 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.740271 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.740631 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.740747 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.743194 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:55733: BlockingRecv error: recv got EOF from 127.6.72.66:55733 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 00:43:37.743439 15445 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 00:43:37.839377 15448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:37.839752 15448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:37.839838 15448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:37.844220 15448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:37.844380 15448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:37.844481 15448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:37.850247 15448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/krb5kdc/oryx_127.6.72.67.keytab
--principal=oryx/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:44679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=45731
--tserver_master_addrs=127.6.72.126:35957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:37.851451 15448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:37.852540 15448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:37.862664 15456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:37.862670 15453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:37.862721 15454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:37.862972 15448 server_base.cc:1034] running on GCE node
I20260925 00:43:37.863445 15448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:37.864130 15448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:37.865321 15448 hybrid_clock.cc:648] HybridClock initialized: now 1790297017865279 us; error 57 us; skew 500 ppm
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297017, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:37.868287 15448 init.cc:377] Logged in from keytab as oryx/127.6.72.67@KRBTEST.COM (short username oryx)
I20260925 00:43:37.869272 15448 webserver.cc:460] Webserver started at http://127.6.72.67:45731/ using document root <none> and password file <none>
I20260925 00:43:37.869843 15448 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:37.869913 15448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:37.874357 15448 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:43:37.876775 15463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.878183 15448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:37.878332 15448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "d1a3f35c1ab140f482cf3de90ac8cfa4"
format_stamp: "Formatted at 2026-09-25 00:43:36 on dist-test-slave-w8nh"
server_key: "b12868d2d58ce08d7cf0bcb4a78ec04d"
server_key_iv: "fc7ed810cf1569a1c004d144a42fd4f9"
server_key_version: "encryptionkey@0"
I20260925 00:43:37.878952 15448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:37.905025 15448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:37.908612 15448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:37.908846 15448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:37.909523 15448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:37.910435 15448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:37.910486 15448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.910563 15448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:37.910615 15448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:37.920841 15448 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:44679
I20260925 00:43:37.920866 15576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:44679 every 8 connection(s)
I20260925 00:43:37.922025 15448 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:37.927805 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 15448
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 00:43:37.935832 15316 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:37.924038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:51631 (local address 127.6.72.126:35957)
0925 00:43:37.924204 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.924207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.925542 (+ 1335us) server_negotiation.cc:407] Connection header received
0925 00:43:37.926460 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.926463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.926515 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.926601 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.928187 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.929109 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.931202 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.931538 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.931638 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.935668 (+ 4030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:51631: BlockingRecv error: recv got EOF from 127.6.72.67:51631 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 00:43:37.936597 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.924549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:51631)
0925 00:43:37.925363 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:37.925417 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:37.926261 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:37.926785 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:37.926793 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:37.927233 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:37.927973 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.927994 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.929475 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.929483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.931067 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.931077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.931697 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.931702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.931920 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.932641 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:37.932668 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:37.935510 (+ 2842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":117,"threads_started":1}
W20260925 00:43:37.937166 15577 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:35957 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.72.126:35957: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
I20260925 00:43:37.939497 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.929375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60946 (local address 127.6.72.126:35957)
0925 00:43:37.929568 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.929573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.929588 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:43:37.929628 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.929631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.929686 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.929778 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.930803 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.931451 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.932771 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.933016 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.933111 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.934862 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:37.934869 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:37.934887 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:37.934915 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:37.937052 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:37.937616 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:37.937622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:37.937628 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:37.937787 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:37.938114 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:37.938117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:37.938118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:37.938637 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:37.938794 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:37.939089 (+ 295us) server_negotiation.cc:299] Negotiation successful
0925 00:43:37.939312 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:37.943753 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.937674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:36977)
0925 00:43:37.937867 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:37.937882 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:37.937954 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:37.938207 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:37.938210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:37.938378 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:37.938492 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.938499 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.939509 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.939516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.940341 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.940351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.940927 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.940929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.941000 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.941514 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:37.941527 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:37.943653 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107}
W20260925 00:43:37.943962 15316 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:37.937773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:36977 (local address 127.6.72.126:35957)
0925 00:43:37.937905 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.937910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.937925 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:43:37.938024 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.938029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.938090 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.938178 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.938614 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.939400 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.940453 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.940810 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.940923 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.943810 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:36977: BlockingRecv error: recv got EOF from 127.6.72.67:36977 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:37 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:37.949613 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:37.944447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:40007)
0925 00:43:37.944546 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:37.944590 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:37.944674 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:37.944928 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:37.944931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:37.945101 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:37.945174 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.945179 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.945951 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.945959 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.946526 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:37.946534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.947022 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.947026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:37.947114 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.947554 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:37.947565 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:37.949442 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 00:43:37.949678 15316 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:37.944508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:40007 (local address 127.6.72.126:35957)
0925 00:43:37.944671 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:37.944676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:37.944690 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:37.944771 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:37.944774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:37.944816 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:37.944914 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:37.945291 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.945838 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.946654 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:37.946906 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:37.947023 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:37.949563 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:40007: BlockingRecv error: recv got EOF from 127.6.72.67:40007 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260925 00:43:37.949999 15577 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 00:43:38 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:38 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:38.557394 15581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:38.551407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:39601)
0925 00:43:38.551704 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:38.551726 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:38.551820 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:38.552117 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:38.552119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:38.552363 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:38.552458 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:38.552464 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.553252 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.553254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:38.553836 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:38.553844 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.554292 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.554294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:38.554356 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:38.554959 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:38.554973 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:38.557207 (+ 2234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":130,"threads_started":1}
W20260925 00:43:38.557466 15582 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:38.551598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:39601 (local address 127.6.72.126:35957)
0925 00:43:38.551845 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:38.551848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:38.551862 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:43:38.551906 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:38.551909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:38.551950 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:38.552057 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:38.552605 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.553147 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.553955 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.554181 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.554264 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:38.557336 (+ 3072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:39601: BlockingRecv error: recv got EOF from 127.6.72.65:39601 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":102,"threads_started":1}
Sep 25 00:43:38 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:38 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:38.750350 15583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:38.744228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:49065)
0925 00:43:38.744578 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:38.744591 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:38.744707 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:38.744985 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:38.744987 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:38.745171 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:38.745271 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:38.745277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.746055 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.746059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:38.746798 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:38.746809 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.747397 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.747401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:38.747484 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:38.748110 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:38.748122 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:38.750217 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":92,"threads_started":1}
W20260925 00:43:38.750499 15582 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:38.744368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:49065 (local address 127.6.72.126:35957)
0925 00:43:38.744517 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:38.744520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:38.744712 (+ 192us) server_negotiation.cc:407] Connection header received
0925 00:43:38.744776 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:38.744778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:38.744826 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:38.744911 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:38.745398 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.745926 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.746925 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.747264 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.747391 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:38.750379 (+ 2988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:49065: BlockingRecv error: recv got EOF from 127.6.72.66:49065 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 00:43:38 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:38 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:38.956127 15584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:38.950705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:37023)
0925 00:43:38.950963 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:38.950979 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:38.951108 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:38.951368 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:38.951371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:38.951564 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:38.951667 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:38.951673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.952501 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.952504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:38.953119 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:38.953127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.953570 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.953573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:38.953639 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:38.954168 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:38.954179 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:38.955988 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
W20260925 00:43:38.956441 15582 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:38.950743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:37023 (local address 127.6.72.126:35957)
0925 00:43:38.950881 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:38.950885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:38.951133 (+ 248us) server_negotiation.cc:407] Connection header received
0925 00:43:38.951186 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:38.951189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:38.951240 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:38.951331 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:38.951801 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.952400 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.953242 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:38.953466 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:38.953549 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:38.956160 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:37023: BlockingRecv error: recv got EOF from 127.6.72.67:37023 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 25 00:43:39 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:39 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:39.564291 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:39.558414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:46355)
0925 00:43:39.558762 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:39.558776 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:39.558874 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:39.559205 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:39.559208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:39.559392 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:39.559486 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:39.559491 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.560216 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.560219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:39.560800 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:39.560808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.561390 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.561395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:39.561491 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:39.562124 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:39.562136 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:39.564099 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":91,"threads_started":1}
W20260925 00:43:39.564549 15586 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:39.558501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:46355 (local address 127.6.72.126:35957)
0925 00:43:39.558831 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:39.558835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:39.558851 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:39.559010 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:39.559013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:39.559064 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:39.559148 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:39.559588 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.560125 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.560933 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.561260 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.561376 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:39.564390 (+ 3014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:46355: BlockingRecv error: recv got EOF from 127.6.72.65:46355 (error 108)
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":148,"threads_started":1}
Sep 25 00:43:39 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:39 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:39.756901 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:39.751310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:49917)
0925 00:43:39.751570 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:39.751584 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:39.751719 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:39.751989 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:39.751993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:39.752197 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:39.752321 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:39.752327 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.753134 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.753140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:39.753695 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:39.753704 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.754165 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.754168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:39.754256 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:39.754758 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:39.754770 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:39.756761 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":81,"threads_started":1}
W20260925 00:43:39.757184 15586 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:39.751359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:49917 (local address 127.6.72.126:35957)
0925 00:43:39.751503 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:39.751508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:39.751665 (+ 157us) server_negotiation.cc:407] Connection header received
0925 00:43:39.751820 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:39.751823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:39.751876 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:39.751961 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:39.752472 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.753020 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.753816 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.754051 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.754159 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:39.756905 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:49917: BlockingRecv error: recv got EOF from 127.6.72.66:49917 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 00:43:39 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:39 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:39.966413 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:39.957017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:37887)
0925 00:43:39.957298 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:39.957312 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:39.957399 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:39.957642 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:39.957644 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:39.957821 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:39.957914 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:39.957919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.958662 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.958665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:39.959218 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:39.959224 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.959638 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.959641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:39.959701 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:39.964311 (+ 4610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:39.964327 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:39.966265 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":105,"threads_started":1}
W20260925 00:43:39.966646 15586 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:39.957194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:37887 (local address 127.6.72.126:35957)
0925 00:43:39.957354 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:39.957359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:39.957379 (+ 20us) server_negotiation.cc:407] Connection header received
0925 00:43:39.957486 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:39.957488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:39.957533 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:39.957609 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:39.958032 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.958566 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.959322 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:39.959533 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:39.959613 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:39.966411 (+ 6798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:37887: BlockingRecv error: recv got EOF from 127.6.72.67:37887 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 25 00:43:40 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:40 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:40.574101 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:40.565254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:42587)
0925 00:43:40.565515 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:40.565529 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:40.565623 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:40.565978 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:40.565980 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:40.566163 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:40.566260 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:40.566265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.567097 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.567099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:40.567723 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:40.567733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.568535 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.568559 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:40.568669 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:40.572035 (+ 3366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:40.572047 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:40.573956 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
W20260925 00:43:40.574286 15590 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:40.565330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:42587 (local address 127.6.72.126:35957)
0925 00:43:40.565600 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:40.565603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:40.565619 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:40.565774 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:40.565777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:40.565830 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:40.565941 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:40.566408 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.566958 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.567954 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.568379 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.568495 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:40.574115 (+ 5620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:42587: BlockingRecv error: recv got EOF from 127.6.72.65:42587 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":100,"threads_started":1}
Sep 25 00:43:40 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:40 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:40.763530 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:40.757788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:46597)
0925 00:43:40.758106 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:40.758122 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:40.758247 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:40.758536 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:40.758539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:40.758745 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:40.758840 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:40.758846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.759707 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.759711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:40.760458 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:40.760469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.761041 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.761043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:40.761112 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:40.761602 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:40.761613 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:40.763388 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":86,"threads_started":1}
W20260925 00:43:40.763665 15590 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:40.757911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:46597 (local address 127.6.72.126:35957)
0925 00:43:40.758094 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:40.758099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:40.758194 (+ 95us) server_negotiation.cc:407] Connection header received
0925 00:43:40.758377 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:40.758380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:40.758421 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:40.758505 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:40.758965 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.759571 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.760581 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.760927 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.761026 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:40.763518 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:46597: BlockingRecv error: recv got EOF from 127.6.72.66:46597 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 25 00:43:40 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:40 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:40.973567 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:40.967243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:46625)
0925 00:43:40.967669 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:40.967686 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:40.967800 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:40.968081 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:40.968083 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:40.968338 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:40.968434 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:40.968440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.969203 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.969206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:40.969888 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:40.969897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.970420 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.970423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:40.970504 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:40.971119 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:40.971131 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:40.973360 (+ 2229us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":88,"threads_started":1}
W20260925 00:43:40.973594 15590 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:40.967486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:46625 (local address 127.6.72.126:35957)
0925 00:43:40.967671 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:40.967677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:40.967748 (+ 71us) server_negotiation.cc:407] Connection header received
0925 00:43:40.967919 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:40.967922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:40.967976 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:40.968062 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:40.968552 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.969086 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.970011 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:40.970310 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:40.970416 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:40.973493 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:46625: BlockingRecv error: recv got EOF from 127.6.72.67:46625 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 25 00:43:41 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:41 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:41.580026 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:41.574915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:44337)
0925 00:43:41.575228 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:41.575241 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:41.575340 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:41.575631 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:41.575633 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:41.575806 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:41.575900 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:41.575905 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.576642 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.576644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:41.577176 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:41.577183 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.577586 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.577588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:41.577654 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:41.578137 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:41.578149 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:41.579913 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":141,"threads_started":1}
W20260925 00:43:41.580215 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:41.574953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:44337 (local address 127.6.72.126:35957)
0925 00:43:41.575243 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:41.575247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:41.575293 (+ 46us) server_negotiation.cc:407] Connection header received
0925 00:43:41.575453 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:41.575455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:41.575503 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:41.575592 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:41.575990 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.576555 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.577269 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.577486 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.577573 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:41.580064 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:44337: BlockingRecv error: recv got EOF from 127.6.72.65:44337 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
Sep 25 00:43:41 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:41 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:41.769730 15595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:41.764395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:53025)
0925 00:43:41.764678 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:41.764691 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:41.764798 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:41.765029 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:41.765031 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:41.765241 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:41.765343 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:41.765348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.766065 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.766067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:41.766636 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:41.766644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.767056 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.767058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:41.767140 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:41.767707 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:41.767719 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:41.769566 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":81,"threads_started":1}
W20260925 00:43:41.769850 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:41.764466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:53025 (local address 127.6.72.126:35957)
0925 00:43:41.764635 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:41.764640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:41.764748 (+ 108us) server_negotiation.cc:407] Connection header received
0925 00:43:41.764868 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:41.764870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:41.764914 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:41.764990 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:41.765443 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.765974 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.766738 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.766956 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.767034 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:41.769723 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:53025: BlockingRecv error: recv got EOF from 127.6.72.66:53025 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 00:43:41 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:41 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:41.979722 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:41.974475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:37655)
0925 00:43:41.974736 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:41.974749 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:41.974853 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:41.975166 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:41.975168 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:41.975339 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:41.975431 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:41.975436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.976216 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.976218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:41.976839 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:41.976848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.977266 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.977268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:41.977327 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:41.977816 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:41.977827 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:41.979607 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":82,"threads_started":1}
W20260925 00:43:41.979844 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:41.974560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:37655 (local address 127.6.72.126:35957)
0925 00:43:41.974737 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:41.974740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:41.974818 (+ 78us) server_negotiation.cc:407] Connection header received
0925 00:43:41.974984 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:41.974987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:41.975041 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:41.975125 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:41.975539 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.976073 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.976943 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:41.977169 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:41.977248 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:41.979728 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:37655: BlockingRecv error: recv got EOF from 127.6.72.67:37655 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 25 00:43:42 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:42 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:42.586472 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:42.580871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:34189)
0925 00:43:42.581165 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:42.581186 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:42.581288 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:42.581569 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:42.581573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:42.581766 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:42.581860 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:42.581866 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.582722 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.582725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:42.583352 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:42.583361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.583831 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.583834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:42.583905 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:42.584425 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:42.584436 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:42.586354 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":87,"threads_started":1}
W20260925 00:43:42.586707 15598 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:42.581022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:34189 (local address 127.6.72.126:35957)
0925 00:43:42.581267 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:42.581271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:42.581284 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:43:42.581369 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:42.581371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:42.581420 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:42.581502 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:42.582004 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.582609 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.583457 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.583722 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.583818 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:42.586544 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:34189: BlockingRecv error: recv got EOF from 127.6.72.65:34189 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
Sep 25 00:43:42 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:42 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:42.775979 15599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:42.770561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:51081)
0925 00:43:42.770858 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:42.770890 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:42.770989 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:42.771252 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:42.771254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:42.771437 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:42.771529 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:42.771535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.772366 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.772369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:42.772958 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:42.772967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.773367 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.773369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:42.773429 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:42.773986 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:42.773998 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:42.775858 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":98,"threads_started":1}
W20260925 00:43:42.776124 15598 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:42.770849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:51081 (local address 127.6.72.126:35957)
0925 00:43:42.770985 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:42.770988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:42.771001 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:43:42.771060 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:42.771062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:42.771103 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:42.771244 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:42.771646 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.772207 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.773059 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.773272 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.773350 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:42.776005 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:51081: BlockingRecv error: recv got EOF from 127.6.72.66:51081 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 25 00:43:42 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:42 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:42.985929 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:42.980594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:48327)
0925 00:43:42.980878 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:42.980891 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:42.980982 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:42.981194 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:42.981197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:42.981381 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:42.981473 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:42.981477 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.982196 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.982200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:42.982862 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:42.982872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.983340 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.983343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:42.983403 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:42.983942 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:42.983954 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:42.985811 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":122,"threads_started":1}
W20260925 00:43:42.986101 15598 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:42.980637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:48327 (local address 127.6.72.126:35957)
0925 00:43:42.980795 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:42.980800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:42.980958 (+ 158us) server_negotiation.cc:407] Connection header received
0925 00:43:42.981049 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:42.981051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:42.981094 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:42.981159 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:42.981566 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.982083 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.983007 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:42.983239 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:42.983324 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:42.985963 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:48327: BlockingRecv error: recv got EOF from 127.6.72.67:48327 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 25 00:43:43 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:43 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:43.593055 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:43.587366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:36457)
0925 00:43:43.587688 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:43.587703 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:43.587800 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:43.588156 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:43.588159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:43.588412 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:43.588508 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:43.588514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.589304 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.589308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:43.589854 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:43.589861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.590315 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.590318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:43.590401 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:43.590963 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:43.590974 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:43.592863 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":89,"threads_started":1}
W20260925 00:43:43.593096 15602 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:43.587457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:36457 (local address 127.6.72.126:35957)
0925 00:43:43.587753 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:43.587756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:43.587772 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:43.587920 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:43.587923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:43.587976 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:43.588078 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:43.588641 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.589178 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.589978 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.590205 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.590312 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:43.592984 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:36457: BlockingRecv error: recv got EOF from 127.6.72.65:36457 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":118,"threads_started":1}
Sep 25 00:43:43 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:43 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:43.782123 15603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:43.776863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:40141)
0925 00:43:43.777117 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:43.777131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:43.777228 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:43.777437 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:43.777439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:43.777618 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:43.777760 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:43.777768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.778539 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.778542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:43.779134 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:43.779144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.779562 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.779565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:43.779627 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:43.780196 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:43.780207 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:43.781975 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
W20260925 00:43:43.782197 15602 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:43.776903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:40141 (local address 127.6.72.126:35957)
0925 00:43:43.777053 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:43.777057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:43.777197 (+ 140us) server_negotiation.cc:407] Connection header received
0925 00:43:43.777294 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:43.777296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:43.777338 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:43.777404 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:43.777866 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.778420 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.779250 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.779459 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.779538 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:43.782105 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:40141: BlockingRecv error: recv got EOF from 127.6.72.66:40141 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 00:43:43 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:43 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:43.992601 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:43.986910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:60607)
0925 00:43:43.987216 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:43.987230 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:43.987337 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:43.987569 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:43.987572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:43.987762 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:43.987854 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:43.987858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.988825 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.988830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:43.989595 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:43.989607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.990017 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.990020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:43.990114 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:43.990668 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:43.990686 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:43.992483 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":93,"threads_started":1}
W20260925 00:43:43.992835 15602 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:43.987018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:60607 (local address 127.6.72.126:35957)
0925 00:43:43.987210 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:43.987215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:43.987304 (+ 89us) server_negotiation.cc:407] Connection header received
0925 00:43:43.987416 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:43.987419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:43.987465 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:43.987536 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:43.987981 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.988665 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.989704 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:43.989919 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:43.990002 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:43.992694 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:60607: BlockingRecv error: recv got EOF from 127.6.72.67:60607 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Sep 25 00:43:44 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:44 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:44.601955 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:44.594127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:50217)
0925 00:43:44.594498 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:44.594514 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:44.594625 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:44.594968 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:44.594972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:44.595223 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:44.595344 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:44.595354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.596661 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.596665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:44.597714 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:44.597725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.598432 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.598435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:44.598542 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:44.599169 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:44.599183 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:44.601766 (+ 2583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":90,"threads_started":1}
W20260925 00:43:44.602344 15606 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:44.594174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:50217 (local address 127.6.72.126:35957)
0925 00:43:44.594548 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:44.594553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:44.594584 (+ 31us) server_negotiation.cc:407] Connection header received
0925 00:43:44.594730 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:44.594734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:44.594816 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:44.594919 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:44.595590 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.596509 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.597860 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.598293 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.598424 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:44.601955 (+ 3531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:50217: BlockingRecv error: recv got EOF from 127.6.72.65:50217 (error 108)
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":110,"threads_started":1}
Sep 25 00:43:44 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:44 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:44.789418 15607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:44.783189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:43149)
0925 00:43:44.783717 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:44.783731 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:44.783861 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:44.784183 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:44.784186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:44.784482 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:44.784617 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:44.784624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.785417 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.785422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:44.786011 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:44.786018 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.786480 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.786490 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:44.786577 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:44.787142 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:44.787155 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:44.789240 (+ 2085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":119,"threads_started":1}
W20260925 00:43:44.789476 15606 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:44.783382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:43149 (local address 127.6.72.126:35957)
0925 00:43:44.783615 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:44.783622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:44.783816 (+ 194us) server_negotiation.cc:407] Connection header received
0925 00:43:44.783957 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:44.783961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:44.784043 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:44.784140 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:44.784728 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.785300 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.786126 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.786360 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.786469 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:44.789367 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:43149: BlockingRecv error: recv got EOF from 127.6.72.66:43149 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 25 00:43:44 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:44 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:45.003681 15608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:44.993473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:60973)
0925 00:43:44.993877 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:44.993890 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:44.994052 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:44.994328 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:44.994331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:44.994555 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:44.994646 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:44.994652 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.995450 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.995456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:44.996013 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:44.996021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.996570 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.996574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:44.996660 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:45.001566 (+ 4906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:45.001580 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:45.003551 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":86,"threads_started":1}
W20260925 00:43:45.003875 15606 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:44.993620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:60973 (local address 127.6.72.126:35957)
0925 00:43:44.993789 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:44.993795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:44.993986 (+ 191us) server_negotiation.cc:407] Connection header received
0925 00:43:44.994163 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:44.994166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:44.994219 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:44.994303 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:44.994766 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.995338 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.996149 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:44.996447 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:44.996557 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:45.003702 (+ 7145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:60973: BlockingRecv error: recv got EOF from 127.6.72.67:60973 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 00:43:45 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:45 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:45.609184 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:45.602890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:57759)
0925 00:43:45.603198 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:45.603218 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:45.603313 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:45.603631 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:45.603634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:45.603853 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:45.603995 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:45.604002 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:45.605056 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:45.605060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:45.605638 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:45.605647 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:45.606205 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:45.606208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:45.606279 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:45.606882 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:45.606895 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:45.609021 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":136,"threads_started":1}
W20260925 00:43:45.609412 15610 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:45.603008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:57759 (local address 127.6.72.126:35957)
0925 00:43:45.603272 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:45.603275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:45.603291 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:45.603441 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:45.603444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:45.603488 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:45.603599 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:45.604113 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:45.604888 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:45.605776 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:45.606073 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:45.606203 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:45.609237 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:57759: BlockingRecv error: recv got EOF from 127.6.72.65:57759 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":106,"threads_started":1}
Sep 25 00:43:45 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:45 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 00:43:45.797333 15610 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:45.790370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:39013 (local address 127.6.72.126:35957)
0925 00:43:45.790549 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:45.790556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:45.790643 (+ 87us) server_negotiation.cc:407] Connection header received
0925 00:43:45.790806 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:45.790813 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:45.790879 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:45.791103 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:45.791585 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:45.792971 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:45.793946 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:45.794413 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:45.794541 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:45.797232 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:39013: BlockingRecv error: recv got EOF from 127.6.72.66:39013 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 00:43:45.797349 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:45.790252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:39013)
0925 00:43:45.790570 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:45.790584 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:45.790681 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:45.791131 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:45.791133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:45.791324 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:45.791431 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:45.791438 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:45.793112 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:45.793115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:45.793837 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:45.793846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:45.794537 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:45.794540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:45.794600 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:45.795133 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:45.795145 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:45.797122 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":77,"threads_started":1}
Sep 25 00:43:46 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:46 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:46.010272 15612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:46.004513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:59327)
0925 00:43:46.004859 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:46.004877 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:46.005028 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:46.005305 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:46.005308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:46.005513 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:46.005601 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:46.005606 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.006408 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.006413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:46.006979 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:46.006987 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.007450 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.007454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:46.007539 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:46.008195 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:46.008206 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:46.010111 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":88,"threads_started":1}
W20260925 00:43:46.010373 15610 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:46.004653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:59327 (local address 127.6.72.126:35957)
0925 00:43:46.004859 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:46.004864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:46.004968 (+ 104us) server_negotiation.cc:407] Connection header received
0925 00:43:46.005137 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:46.005140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:46.005191 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:46.005283 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:46.005750 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.006303 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.007100 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.007338 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.007446 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:46.010261 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:59327: BlockingRecv error: recv got EOF from 127.6.72.67:59327 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 25 00:43:46 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:46 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:46.616024 15613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:46.609964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:42755)
0925 00:43:46.610295 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:46.610342 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:46.610541 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:46.610887 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:46.610890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:46.611121 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:46.611227 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:46.611233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.611942 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.611945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:46.612576 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:46.612584 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.613003 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.613006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:46.613066 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:46.613629 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:46.613653 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:46.615909 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
W20260925 00:43:46.616168 15614 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:46.610094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:42755 (local address 127.6.72.126:35957)
0925 00:43:46.610374 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:46.610380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:46.610425 (+ 45us) server_negotiation.cc:407] Connection header received
0925 00:43:46.610666 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:46.610670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:46.610741 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:46.610849 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:46.611320 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.611844 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.612690 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.612911 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.612991 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:46.616042 (+ 3051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:42755: BlockingRecv error: recv got EOF from 127.6.72.65:42755 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
Sep 25 00:43:46 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:46 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:46.804409 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:46.798300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:43517)
0925 00:43:46.798593 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:46.798607 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:46.798753 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:46.799026 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:46.799028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:46.799236 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:46.799335 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:46.799341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.800099 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.800104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:46.800777 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:46.800786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.801223 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.801227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:46.801305 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:46.801854 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:46.801866 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:46.804207 (+ 2341us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":93,"threads_started":1}
W20260925 00:43:46.804504 15614 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:46.798368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:43517 (local address 127.6.72.126:35957)
0925 00:43:46.798535 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:46.798540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:46.798694 (+ 154us) server_negotiation.cc:407] Connection header received
0925 00:43:46.798872 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:46.798875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:46.798919 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:46.799005 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:46.799452 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.799993 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.800888 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:46.801107 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:46.801221 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:46.804368 (+ 3147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:43517: BlockingRecv error: recv got EOF from 127.6.72.66:43517 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 25 00:43:47 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:47 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:47.016587 15616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:47.011111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:34955)
0925 00:43:47.011393 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:47.011406 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:47.011506 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:47.011768 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:47.011770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:47.011946 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:47.012045 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:47.012050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.012883 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.012885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:47.013456 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:47.013465 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.013871 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.013874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:47.013931 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:47.014418 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:47.014430 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:47.016443 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":90,"threads_started":1}
W20260925 00:43:47.016716 15614 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:47.011255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:34955 (local address 127.6.72.126:35957)
0925 00:43:47.011413 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:47.011417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:47.011458 (+ 41us) server_negotiation.cc:407] Connection header received
0925 00:43:47.011590 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:47.011592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:47.011638 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:47.011709 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:47.012176 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.012769 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.013576 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.013781 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.013856 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:47.016590 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:34955: BlockingRecv error: recv got EOF from 127.6.72.67:34955 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 25 00:43:47 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:47 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:47.622320 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:47.616910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:42677)
0925 00:43:47.617231 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:47.617244 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:47.617335 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:47.617563 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:47.617565 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:47.617763 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:47.617859 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:47.617864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.618618 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.618620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:47.619198 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:47.619205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.619683 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.619685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:47.619742 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:47.620235 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:47.620272 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:47.622165 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":89,"threads_started":1}
W20260925 00:43:47.622434 15618 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:47.617044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:42677 (local address 127.6.72.126:35957)
0925 00:43:47.617315 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:47.617319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:47.617332 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:43:47.617408 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:47.617410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:47.617448 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:47.617529 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:47.617972 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.618527 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.619319 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.619580 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.619672 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:47.622275 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:42677: BlockingRecv error: recv got EOF from 127.6.72.65:42677 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":96,"threads_started":1}
Sep 25 00:43:47 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:47 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:47.810549 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:47.805233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:40497)
0925 00:43:47.805513 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:47.805527 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:47.805619 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:47.805859 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:47.805861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:47.806054 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:47.806150 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:47.806156 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.806859 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.806861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:47.807418 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:47.807426 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.807812 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.807814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:47.807874 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:47.808455 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:47.808468 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:47.810372 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":102,"threads_started":1}
W20260925 00:43:47.810611 15618 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:47.805309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:40497 (local address 127.6.72.126:35957)
0925 00:43:47.805439 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:47.805442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:47.805591 (+ 149us) server_negotiation.cc:407] Connection header received
0925 00:43:47.805697 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:47.805700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:47.805741 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:47.805815 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:47.806247 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.806763 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.807515 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:47.807722 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:47.807793 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:47.810504 (+ 2711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:40497: BlockingRecv error: recv got EOF from 127.6.72.66:40497 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 00:43:48 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:48 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:48.023144 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:48.017538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:60553)
0925 00:43:48.017873 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:48.017890 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:48.017998 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:48.018302 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:48.018305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:48.018540 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:48.018631 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:48.018637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.019382 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.019388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:48.019951 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:48.019959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.020448 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.020452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:48.020545 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:48.021127 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:48.021139 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:48.022949 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":137,"threads_started":1}
W20260925 00:43:48.023185 15618 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:48.017711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:60553 (local address 127.6.72.126:35957)
0925 00:43:48.017867 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:48.017872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:48.017948 (+ 76us) server_negotiation.cc:407] Connection header received
0925 00:43:48.018136 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:48.018138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:48.018184 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:48.018278 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:48.018749 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.019276 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.020068 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.020341 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.020445 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:48.023080 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:60553: BlockingRecv error: recv got EOF from 127.6.72.67:60553 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 25 00:43:48 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:48 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:48.628650 15621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:48.623208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:36865)
0925 00:43:48.623573 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:48.623618 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:48.623757 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:48.624163 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:48.624168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:48.624372 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:48.624468 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:48.624474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.625199 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.625201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:48.625739 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:48.625746 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.626137 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.626140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:48.626200 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:48.626663 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:48.626674 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:48.628523 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":78,"threads_started":1}
W20260925 00:43:48.628997 15622 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:48.623340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:36865 (local address 127.6.72.126:35957)
0925 00:43:48.623606 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:48.623613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:48.623704 (+ 91us) server_negotiation.cc:407] Connection header received
0925 00:43:48.623882 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:48.623888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:48.623964 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:48.624076 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:48.624572 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.625093 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.625832 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.626043 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.626115 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:48.628768 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:36865: BlockingRecv error: recv got EOF from 127.6.72.65:36865 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
Sep 25 00:43:48 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:48 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:48.817231 15623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:48.811497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:46935)
0925 00:43:48.811801 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:48.811817 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:48.811963 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:48.812262 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:48.812266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:48.812497 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:48.812599 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:48.812605 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.813357 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.813364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:48.813910 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:48.813938 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.814480 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.814484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:48.814571 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:48.815054 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:48.815066 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:48.817046 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":82,"threads_started":1}
W20260925 00:43:48.817337 15622 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:48.811577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:46935 (local address 127.6.72.126:35957)
0925 00:43:48.811766 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:48.811771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:48.811900 (+ 129us) server_negotiation.cc:407] Connection header received
0925 00:43:48.812062 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:48.812066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:48.812119 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:48.812211 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:48.812721 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.813248 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.814071 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:48.814363 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:48.814477 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:48.817215 (+ 2738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:46935: BlockingRecv error: recv got EOF from 127.6.72.66:46935 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 25 00:43:49 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:49 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:49.029925 15624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:49.024145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:51503)
0925 00:43:49.024508 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:49.024529 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:49.024660 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:49.024924 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:49.024927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:49.025162 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:49.025252 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:49.025258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.026024 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.026029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:49.026583 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:49.026592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.027035 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.027039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:49.027131 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:49.027700 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:49.027712 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:49.029750 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":130,"threads_started":1}
W20260925 00:43:49.029979 15622 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:49.024294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:51503 (local address 127.6.72.126:35957)
0925 00:43:49.024461 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:49.024466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:49.024639 (+ 173us) server_negotiation.cc:407] Connection header received
0925 00:43:49.024767 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:49.024770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:49.024815 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:49.024900 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:49.025375 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.025913 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.026703 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.026918 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.027030 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:49.029873 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:51503: BlockingRecv error: recv got EOF from 127.6.72.67:51503 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 25 00:43:49 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:49 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:49.634613 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:49.629372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:51095)
0925 00:43:49.629643 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:49.629656 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:49.629744 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:49.629992 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:49.629994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:49.630179 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:49.630274 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:49.630279 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.631055 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.631057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:49.631605 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:49.631612 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.632084 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.632086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:49.632146 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:49.632698 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:49.632709 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:49.634499 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":102,"threads_started":1}
W20260925 00:43:49.634748 15626 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:49.629496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:51095 (local address 127.6.72.126:35957)
0925 00:43:49.629751 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:49.629755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:49.629793 (+ 38us) server_negotiation.cc:407] Connection header received
0925 00:43:49.629831 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:49.629833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:49.629876 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:49.629952 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:49.630377 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.630959 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.631727 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.631965 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.632070 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:49.634618 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:51095: BlockingRecv error: recv got EOF from 127.6.72.65:51095 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":104,"threads_started":1}
Sep 25 00:43:49 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:49 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:49.823192 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:49.817917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:35075)
0925 00:43:49.818207 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:49.818220 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:49.818317 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:49.818563 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:49.818565 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:49.818743 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:49.818865 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:49.818871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.819626 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.819629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:49.820180 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:49.820188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.820640 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.820643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:49.820717 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:49.821278 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:49.821290 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:49.823076 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":95,"threads_started":1}
W20260925 00:43:49.823354 15626 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:49.818013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:35075 (local address 127.6.72.126:35957)
0925 00:43:49.818155 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:49.818159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:49.818289 (+ 130us) server_negotiation.cc:407] Connection header received
0925 00:43:49.818396 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:49.818398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:49.818441 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:49.818514 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:49.818980 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.819522 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.820329 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:49.820543 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:49.820631 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:49.823226 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:35075: BlockingRecv error: recv got EOF from 127.6.72.66:35075 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 25 00:43:50 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:50 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:50.039402 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:50.030769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:44143)
0925 00:43:50.031068 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:50.031081 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:50.031179 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:50.031399 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:50.031401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:50.031594 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:50.031692 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:50.031697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.032465 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.032468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:50.033022 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:50.033031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.033450 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.033452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:50.033527 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:50.037108 (+ 3581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:50.037121 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:50.039246 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":122,"threads_started":1}
W20260925 00:43:50.039506 15626 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:50.030871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:44143 (local address 127.6.72.126:35957)
0925 00:43:50.031003 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:50.031007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:50.031156 (+ 149us) server_negotiation.cc:407] Connection header received
0925 00:43:50.031245 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:50.031247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:50.031291 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:50.031359 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:50.031790 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.032358 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.033121 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.033345 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.033424 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:50.039381 (+ 5957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:44143: BlockingRecv error: recv got EOF from 127.6.72.67:44143 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 00:43:50 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:50 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:50.640827 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:50.635468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:47979)
0925 00:43:50.635755 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:50.635769 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:50.635857 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:50.636108 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:50.636110 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:50.636321 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:50.636416 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:50.636421 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.637160 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.637163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:50.637757 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:50.637765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.638168 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.638171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:50.638236 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:50.638735 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:50.638746 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:50.640692 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":108,"threads_started":1}
W20260925 00:43:50.640980 15630 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:50.635559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:47979 (local address 127.6.72.126:35957)
0925 00:43:50.635794 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:50.635797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:50.635812 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:43:50.635943 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:50.635945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:50.635986 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:50.636064 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:50.636526 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.637058 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.637855 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.638065 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.638138 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:50.640840 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:47979: BlockingRecv error: recv got EOF from 127.6.72.65:47979 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":92,"threads_started":1}
Sep 25 00:43:50 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:50 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:50.829635 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:50.824008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:59251)
0925 00:43:50.824349 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:50.824366 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:50.824488 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:50.824775 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:50.824778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:50.824990 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:50.825085 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:50.825092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.825852 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.825858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:50.826414 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:50.826422 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.826891 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.826895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:50.826980 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:50.827549 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:50.827560 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:50.829502 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
W20260925 00:43:50.829890 15630 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:50.824185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:59251 (local address 127.6.72.126:35957)
0925 00:43:50.824349 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:50.824354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:50.824445 (+ 91us) server_negotiation.cc:407] Connection header received
0925 00:43:50.824597 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:50.824603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:50.824661 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:50.824747 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:50.825211 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.825743 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.826538 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:50.826769 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:50.826877 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:50.829659 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:59251: BlockingRecv error: recv got EOF from 127.6.72.66:59251 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 25 00:43:51 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:51 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:51.045514 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:51.040307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:58183)
0925 00:43:51.040563 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:51.040577 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:51.040705 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:51.040937 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:51.040940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:51.041122 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:51.041214 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:51.041219 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.041949 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.041951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:51.042497 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:51.042504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.042919 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.042921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:51.042977 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:51.043481 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:51.043492 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:51.045392 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":80,"threads_started":1}
W20260925 00:43:51.045714 15630 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:51.040355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:58183 (local address 127.6.72.126:35957)
0925 00:43:51.040501 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:51.040505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:51.040646 (+ 141us) server_negotiation.cc:407] Connection header received
0925 00:43:51.040789 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:51.040791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:51.040839 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:51.040901 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:51.041321 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.041849 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.042610 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.042822 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.042896 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:51.045569 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:58183: BlockingRecv error: recv got EOF from 127.6.72.67:58183 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 25 00:43:51 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:51 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:51.647425 15633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:51.641733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:36599)
0925 00:43:51.642011 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:51.642025 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:51.642118 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:51.642433 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:51.642436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:51.642630 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:51.642745 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:51.642750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.643580 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.643583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:51.644160 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:51.644169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.644675 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.644678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:51.644744 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:51.645352 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:51.645364 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:51.647287 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":89,"threads_started":1}
W20260925 00:43:51.647610 15634 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:51.641771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:36599 (local address 127.6.72.126:35957)
0925 00:43:51.642067 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:51.642070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:51.642109 (+ 39us) server_negotiation.cc:407] Connection header received
0925 00:43:51.642228 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:51.642230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:51.642275 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:51.642355 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:51.642903 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.643453 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.644287 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.644562 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.644649 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:51.647459 (+ 2810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:36599: BlockingRecv error: recv got EOF from 127.6.72.65:36599 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":55,"threads_started":1}
Sep 25 00:43:51 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:51 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:51.836803 15635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:51.830603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:43585)
0925 00:43:51.830916 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:51.830929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:51.831037 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:51.831275 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:51.831277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:51.831457 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:51.831550 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:51.831555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.832572 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.832579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:51.833279 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:51.833290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.833910 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.833913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:51.834008 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:51.834558 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:51.834570 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:51.836656 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":95,"threads_started":1}
W20260925 00:43:51.836885 15634 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:51.830690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:43585 (local address 127.6.72.126:35957)
0925 00:43:51.830856 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:51.830862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:51.830990 (+ 128us) server_negotiation.cc:407] Connection header received
0925 00:43:51.831112 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:51.831114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:51.831161 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:51.831233 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:51.831690 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.832381 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.833509 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:51.833778 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:51.833903 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:51.836791 (+ 2888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:43585: BlockingRecv error: recv got EOF from 127.6.72.66:43585 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 25 00:43:52 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:52 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.67@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:52.052389 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:52.046490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.67:39307)
0925 00:43:52.046863 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:52.046875 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:52.047013 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:52.047308 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:52.047311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:52.047523 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:52.047612 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:52.047633 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.048464 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.048469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:52.049039 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:52.049047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.049510 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.049514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:52.049598 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:52.050203 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:52.050215 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:52.052195 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":108,"threads_started":1}
W20260925 00:43:52.052502 15634 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:52.046637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:39307 (local address 127.6.72.126:35957)
0925 00:43:52.046811 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:52.046815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:52.046955 (+ 140us) server_negotiation.cc:407] Connection header received
0925 00:43:52.047143 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:52.047145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:52.047192 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:52.047286 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:52.047751 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.048355 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.049167 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.049393 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.049506 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:52.052387 (+ 2881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.67:39307: BlockingRecv error: recv got EOF from 127.6.72.67:39307 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 00:43:52 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:52 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.65@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:52.654212 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:52.648560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.65:41201)
0925 00:43:52.648885 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:52.648899 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:52.648992 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:52.649279 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:52.649281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:52.649496 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:52.649597 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:52.649602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.650388 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.650391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:52.650977 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:52.650985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.651436 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.651438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:52.651511 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:52.652022 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:52.652036 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:52.654087 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":88,"threads_started":1}
W20260925 00:43:52.654441 15638 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:52.648632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:41201 (local address 127.6.72.126:35957)
0925 00:43:52.648922 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:52.648925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:52.648943 (+ 18us) server_negotiation.cc:407] Connection header received
0925 00:43:52.649082 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:52.649085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:52.649145 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:52.649225 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:52.649713 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.650276 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.651086 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.651333 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.651416 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:52.654283 (+ 2867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.65:41201: BlockingRecv error: recv got EOF from 127.6.72.65:41201 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":111,"threads_started":1}
Sep 25 00:43:52 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 00:43:52 dist-test-slave-w8nh krb5kdc[14580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.72.66@KRBTEST.COM for oryx/127.6.72.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 00:43:52.843199 15639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:52.837793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:35957 (local address 127.6.72.66:47875)
0925 00:43:52.838097 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:52.838110 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:52.838201 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:52.838424 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:52.838426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:52.838618 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:52.838711 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:52.838716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.839485 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.839487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:52.840047 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:52.840055 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.840547 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.840559 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:52.840620 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:52.841118 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:52.841130 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:52.843002 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.72.126:35957: Server oryx/127.6.72.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":135,"threads_started":1}
W20260925 00:43:52.843256 15638 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:52.837884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:47875 (local address 127.6.72.126:35957)
0925 00:43:52.838072 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:52.838079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:52.838186 (+ 107us) server_negotiation.cc:407] Connection header received
0925 00:43:52.838270 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:52.838272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:52.838319 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:52.838397 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:52.838843 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.839388 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.840153 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:52.840450 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:52.840534 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:52.843129 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.72.66:47875: BlockingRecv error: recv got EOF from 127.6.72.66:47875 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260925 00:43:52.939244 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 15163
I20260925 00:43:52.945351 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 15315
I20260925 00:43:52.950663 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 15448
I20260925 00:43:52.956362 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 15093
2026-09-25T00:43:52Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18087 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T00:43:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:52Z Disabled control of system clock
I20260925 00:43:52.987318 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:42473
--webserver_interface=127.6.72.126
--webserver_port=0
--builtin_ntp_servers=127.6.72.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:42473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 00:43:53.100342 15643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:53.100646 15643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:53.100714 15643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:53.104938 15643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:53.105000 15643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:53.105015 15643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:53.105032 15643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:53.105048 15643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:53.110225 15643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:42473
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:42473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:53.111321 15643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:53.112808 15643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:53.121130 15649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:53.121140 15651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:53.121140 15648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:53.121464 15643 server_base.cc:1034] running on GCE node
I20260925 00:43:53.122071 15643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:53.123234 15643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:53.124392 15643 hybrid_clock.cc:648] HybridClock initialized: now 1790297033124360 us; error 56 us; skew 500 ppm
I20260925 00:43:53.126609 15643 webserver.cc:460] Webserver started at http://127.6.72.126:45517/ using document root <none> and password file <none>
I20260925 00:43:53.127509 15643 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:53.127622 15643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:53.127874 15643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:53.131083 15643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "e74b5d3274e7459da1c95b66b2d413c6"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "ef60dcad62e99229e433e61e6a00fc88"
server_key_iv: "0c6d58b084f7f587126c14ca8aa8e8c0"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.131870 15643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "e74b5d3274e7459da1c95b66b2d413c6"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "ef60dcad62e99229e433e61e6a00fc88"
server_key_iv: "0c6d58b084f7f587126c14ca8aa8e8c0"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.136206 15643 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 00:43:53.138921 15657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:53.140331 15643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:53.140478 15643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "e74b5d3274e7459da1c95b66b2d413c6"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "ef60dcad62e99229e433e61e6a00fc88"
server_key_iv: "0c6d58b084f7f587126c14ca8aa8e8c0"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.140643 15643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:53.157940 15643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:53.158738 15643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:53.158957 15643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:53.166507 15643 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:42473
I20260925 00:43:53.166580 15709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:42473 every 8 connection(s)
I20260925 00:43:53.167580 15643 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:53.171242 15710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:53.173358 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 15643
I20260925 00:43:53.173489 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:53.173784 6433 external_mini_cluster.cc:1428] Setting key c54af68748c3b803ce19cc34402ad6a2
I20260925 00:43:53.177635 15710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6: Bootstrap starting.
I20260925 00:43:53.180203 15710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:53.181140 15710 log.cc:824] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:53.183424 15710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6: No bootstrap required, opened a new log
I20260925 00:43:53.184051 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:53.175327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.6.72.126:42473)
0925 00:43:53.176058 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:53.176075 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:53.176114 (+ 39us) server_negotiation.cc:407] Connection header received
0925 00:43:53.176878 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:53.176897 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:53.177337 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:53.177636 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:53.178136 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.179252 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.180546 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.181036 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.181189 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:53.182260 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:53.182266 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:53.182278 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:53.182665 (+ 387us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:53.182721 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:53.182741 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:53.182817 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:53.182985 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 00:43:53.183296 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":170,"threads_started":1}
I20260925 00:43:53.187381 15710 raft_consensus.cc:348] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74b5d3274e7459da1c95b66b2d413c6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42473 } }
I20260925 00:43:53.187597 15710 raft_consensus.cc:374] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:53.187678 15710 raft_consensus.cc:729] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e74b5d3274e7459da1c95b66b2d413c6, State: Initialized, Role: FOLLOWER
I20260925 00:43:53.188082 15710 consensus_queue.cc:260] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74b5d3274e7459da1c95b66b2d413c6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42473 } }
I20260925 00:43:53.188222 15710 raft_consensus.cc:388] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:53.188301 15710 raft_consensus.cc:482] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:53.188428 15710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:53.189356 15710 raft_consensus.cc:504] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74b5d3274e7459da1c95b66b2d413c6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42473 } }
I20260925 00:43:53.189702 15710 leader_election.cc:302] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e74b5d3274e7459da1c95b66b2d413c6; no voters:
I20260925 00:43:53.190037 15710 leader_election.cc:288] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:53.190124 15715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:53.190362 15715 raft_consensus.cc:686] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [term 1 LEADER]: Becoming Leader. State: Replica: e74b5d3274e7459da1c95b66b2d413c6, State: Running, Role: LEADER
I20260925 00:43:53.190692 15715 consensus_queue.cc:237] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74b5d3274e7459da1c95b66b2d413c6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42473 } }
I20260925 00:43:53.191314 15710 sys_catalog.cc:564] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:53.192446 15717 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e74b5d3274e7459da1c95b66b2d413c6. Latest consensus state: current_term: 1 leader_uuid: "e74b5d3274e7459da1c95b66b2d413c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74b5d3274e7459da1c95b66b2d413c6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42473 } } }
I20260925 00:43:53.192580 15717 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:53.192843 15716 sys_catalog.cc:455] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e74b5d3274e7459da1c95b66b2d413c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e74b5d3274e7459da1c95b66b2d413c6" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42473 } } }
I20260925 00:43:53.192940 15716 sys_catalog.cc:458] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:53.193074 15724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:53.196417 15724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:53.202504 15724 catalog_manager.cc:1347] Generated new cluster ID: d3235f3b3c99456f8bd628f77c3c2dad
I20260925 00:43:53.202585 15724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:53.217716 15724 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:53.218725 15724 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:53.229398 15724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e74b5d3274e7459da1c95b66b2d413c6: Generated new TSK 0
I20260925 00:43:53.230326 15724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:43:53.241537 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:42473
--builtin_ntp_servers=127.6.72.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 00:43:53.356478 15734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:53.356809 15734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:53.356878 15734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:53.361208 15734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:53.361300 15734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:53.361394 15734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:53.366318 15734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:42473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:53.367810 15734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:53.369329 15734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:53.379233 15739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:53.379266 15740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:53.379233 15742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:53.379694 15734 server_base.cc:1034] running on GCE node
I20260925 00:43:53.380129 15734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:53.380864 15734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:53.382026 15734 hybrid_clock.cc:648] HybridClock initialized: now 1790297033382024 us; error 43 us; skew 500 ppm
I20260925 00:43:53.383994 15734 webserver.cc:460] Webserver started at http://127.6.72.65:34733/ using document root <none> and password file <none>
I20260925 00:43:53.384662 15734 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:53.384727 15734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:53.384948 15734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:53.387065 15734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "6420174437864ed4aed16427dc2b6776"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "66b951904eda3544ffb9ac2c3b288a51"
server_key_iv: "d2825cf5e497b39aa9ad78f595ec7678"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.387676 15734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "6420174437864ed4aed16427dc2b6776"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "66b951904eda3544ffb9ac2c3b288a51"
server_key_iv: "d2825cf5e497b39aa9ad78f595ec7678"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.391572 15734 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:43:53.394542 15748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:53.395850 15734 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:43:53.395987 15734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "6420174437864ed4aed16427dc2b6776"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "66b951904eda3544ffb9ac2c3b288a51"
server_key_iv: "d2825cf5e497b39aa9ad78f595ec7678"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.396107 15734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:53.415228 15734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:53.415977 15734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:53.416201 15734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:53.416888 15734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:53.417886 15734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:53.417959 15734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:53.418033 15734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:53.418083 15734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:53.427556 15734 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:42173
I20260925 00:43:53.427574 15861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:42173 every 8 connection(s)
I20260925 00:43:53.428601 15734 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:53.438159 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:53.430514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:54371 (local address 127.6.72.126:42473)
0925 00:43:53.430653 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:53.430665 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:53.431701 (+ 1036us) server_negotiation.cc:407] Connection header received
0925 00:43:53.433255 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:53.433258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:53.433324 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:53.433425 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:53.434582 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.435175 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.436368 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.436612 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.436705 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:53.437734 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:53.437738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:53.437740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:53.437753 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:53.437772 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:53.437780 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:53.437840 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:53.437979 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 00:43:53.438032 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 00:43:53.438818 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 15734
I20260925 00:43:53.438937 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:53.439241 6433 external_mini_cluster.cc:1428] Setting key 4c937bba64f01f6ed59386061102a07b
I20260925 00:43:53.439275 15864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:53.430858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42473 (local address 127.6.72.65:54371)
0925 00:43:53.431533 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:53.431574 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:53.433011 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:53.433614 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:53.433621 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:53.433751 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:53.434401 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:53.434418 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.435305 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.435308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:53.436189 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:53.436198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.436754 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.436757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:53.436897 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:53.437477 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:53.437478 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:53.437519 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:53.437585 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:53.437853 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:53.437859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:53.437863 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 00:43:53.437955 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 00:43:53.438184 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":120,"threads_started":1}
I20260925 00:43:53.440536 15862 heartbeater.cc:347] Connected to a master server at 127.6.72.126:42473
I20260925 00:43:53.440821 15862 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:53.441483 15862 heartbeater.cc:511] Master 127.6.72.126:42473 requested a full tablet report, sending...
I20260925 00:43:53.441571 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:42473
--builtin_ntp_servers=127.6.72.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 00:43:53.443185 15670 ts_manager.cc:194] Registered new tserver with Master: 6420174437864ed4aed16427dc2b6776 (127.6.72.65:42173)
I20260925 00:43:53.444653 15670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.65:54371
W20260925 00:43:53.556634 15865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:53.556977 15865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:53.557052 15865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:53.561168 15865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:53.561266 15865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:53.561359 15865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:53.566162 15865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:42473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:53.567380 15865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:53.568826 15865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:53.579391 15873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:53.579367 15871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:53.579455 15870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:53.579690 15865 server_base.cc:1034] running on GCE node
I20260925 00:43:53.580363 15865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:53.581082 15865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:53.582273 15865 hybrid_clock.cc:648] HybridClock initialized: now 1790297033582256 us; error 30 us; skew 500 ppm
I20260925 00:43:53.584223 15865 webserver.cc:460] Webserver started at http://127.6.72.66:40337/ using document root <none> and password file <none>
I20260925 00:43:53.584900 15865 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:53.585006 15865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:53.585253 15865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:53.587440 15865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "93d047a79c954c48a1db72e1e97e69ec"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "49b9fa195cd3925a4f40c8556d2f0aa5"
server_key_iv: "6e4c896e0e86166dedbd886cbab494bf"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.588045 15865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "93d047a79c954c48a1db72e1e97e69ec"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "49b9fa195cd3925a4f40c8556d2f0aa5"
server_key_iv: "6e4c896e0e86166dedbd886cbab494bf"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.592131 15865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260925 00:43:53.595116 15879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:53.596582 15865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:53.596750 15865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "93d047a79c954c48a1db72e1e97e69ec"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "49b9fa195cd3925a4f40c8556d2f0aa5"
server_key_iv: "6e4c896e0e86166dedbd886cbab494bf"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.596894 15865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:53.609437 15865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:53.610213 15865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:53.610432 15865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:53.611058 15865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:53.612054 15865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:53.612128 15865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:53.612202 15865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:53.612277 15865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260925 00:43:53.621646 15865 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:34209
I20260925 00:43:53.621666 15992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:34209 every 8 connection(s)
I20260925 00:43:53.622645 15865 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:53.629773 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 15865
I20260925 00:43:53.629882 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:53.630213 6433 external_mini_cluster.cc:1428] Setting key 6393d03376f9b870656ae27f4705208f
I20260925 00:43:53.632414 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:42473
--builtin_ntp_servers=127.6.72.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 00:43:53.632486 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:53.624667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:57371 (local address 127.6.72.126:42473)
0925 00:43:53.624810 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:53.624814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:53.625871 (+ 1057us) server_negotiation.cc:407] Connection header received
0925 00:43:53.627347 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:53.627350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:53.627427 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:53.627509 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:53.628457 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.629008 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.630285 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.630552 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.630670 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:53.631947 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:53.631951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:53.631953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:53.631967 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:53.631989 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:53.632003 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:53.632093 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:53.632280 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 00:43:53.632345 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:43:53.633415 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:53.625023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42473 (local address 127.6.72.66:57371)
0925 00:43:53.625706 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:53.625746 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:53.627111 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:53.627689 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:53.627696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:53.627783 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:53.628305 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:53.628317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.629144 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.629147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:53.630154 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:53.630164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.630716 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.630720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:53.630948 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:53.631667 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:53.631668 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:53.631712 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:53.631780 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:53.632094 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:53.632106 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:53.632114 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 00:43:53.632275 (+ 161us) client_negotiation.cc:240] Negotiation successful
0925 00:43:53.632585 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":117,"threads_started":1}
I20260925 00:43:53.634684 15993 heartbeater.cc:347] Connected to a master server at 127.6.72.126:42473
I20260925 00:43:53.634964 15993 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:53.635609 15993 heartbeater.cc:511] Master 127.6.72.126:42473 requested a full tablet report, sending...
I20260925 00:43:53.636910 15670 ts_manager.cc:194] Registered new tserver with Master: 93d047a79c954c48a1db72e1e97e69ec (127.6.72.66:34209)
I20260925 00:43:53.637521 15670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.66:57371
W20260925 00:43:53.747284 15996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:53.747609 15996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:53.747685 15996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:53.751914 15996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:53.752017 15996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:53.752113 15996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:53.757066 15996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:42473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:53.758219 15996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:53.759652 15996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:53.769410 16004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:53.769629 15996 server_base.cc:1034] running on GCE node
W20260925 00:43:53.769481 16001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:53.769410 16002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:53.770274 15996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:53.770925 15996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:53.772176 15996 hybrid_clock.cc:648] HybridClock initialized: now 1790297033772159 us; error 28 us; skew 500 ppm
I20260925 00:43:53.774313 15996 webserver.cc:460] Webserver started at http://127.6.72.67:43661/ using document root <none> and password file <none>
I20260925 00:43:53.774986 15996 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:53.775097 15996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:53.775334 15996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:53.777678 15996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "31448e182c8146aba7b3efdb85dd706b"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "595a95fee0beb0373d6e6ece9be99426"
server_key_iv: "1e5efbfb37b635c469082645ba451ef2"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.778283 15996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "31448e182c8146aba7b3efdb85dd706b"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "595a95fee0beb0373d6e6ece9be99426"
server_key_iv: "1e5efbfb37b635c469082645ba451ef2"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.782289 15996 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:43:53.785113 16010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:53.786453 15996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 00:43:53.786602 15996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "31448e182c8146aba7b3efdb85dd706b"
format_stamp: "Formatted at 2026-09-25 00:43:53 on dist-test-slave-w8nh"
server_key: "595a95fee0beb0373d6e6ece9be99426"
server_key_iv: "1e5efbfb37b635c469082645ba451ef2"
server_key_version: "encryptionkey@0"
I20260925 00:43:53.786728 15996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:53.799415 15996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:53.800282 15996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:53.800513 15996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:53.801204 15996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:53.802193 15996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:53.802269 15996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:53.802347 15996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:53.802393 15996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:53.811678 15996 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:37775
I20260925 00:43:53.811703 16123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:37775 every 8 connection(s)
I20260925 00:43:53.812700 15996 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:53.819872 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 15996
I20260925 00:43:53.819991 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:53.820322 6433 external_mini_cluster.cc:1428] Setting key 7370bfd4ca949a1d174444e4b1c3be0c
I20260925 00:43:53.822829 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:53.814519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:44461 (local address 127.6.72.126:42473)
0925 00:43:53.814686 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:53.814690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:53.815751 (+ 1061us) server_negotiation.cc:407] Connection header received
0925 00:43:53.817357 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:53.817361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:53.817409 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:53.817486 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:53.818454 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.818974 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.820572 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.820885 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.820991 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:53.822257 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:53.822260 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:53.822263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:53.822276 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:53.822322 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:53.822333 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:53.822406 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:53.822614 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 00:43:53.822676 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 00:43:53.823765 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:53.814907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:42473 (local address 127.6.72.67:44461)
0925 00:43:53.815600 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:53.815642 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:53.817149 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:53.817661 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:53.817668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:53.817760 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:53.818320 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:53.818333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.819106 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.819109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:53.820445 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:53.820457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:53.821072 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:53.821081 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:53.821283 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:53.821987 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:53.821989 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:53.822037 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:53.822105 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:53.822415 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:53.822421 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:53.822426 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 00:43:53.822573 (+ 147us) client_negotiation.cc:240] Negotiation successful
0925 00:43:53.822889 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":121,"threads_started":1}
I20260925 00:43:53.824854 16124 heartbeater.cc:347] Connected to a master server at 127.6.72.126:42473
I20260925 00:43:53.825090 16124 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:53.825608 16124 heartbeater.cc:511] Master 127.6.72.126:42473 requested a full tablet report, sending...
I20260925 00:43:53.826614 15670 ts_manager.cc:194] Registered new tserver with Master: 31448e182c8146aba7b3efdb85dd706b (127.6.72.67:37775)
I20260925 00:43:53.827183 15670 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.67:44461
I20260925 00:43:53.834599 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 00:43:53.837476 15713 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:53.836602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.6.72.126:42473)
0925 00:43:53.836736 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:53.836739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:53.836937 (+ 198us) server_negotiation.cc:407] Connection header received
0925 00:43:53.837030 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:53.837032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:53.837088 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:53.837151 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:53.837322 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58818: BlockingRecv error: recv got EOF from 127.0.0.1:58818 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 00:43:53.837699 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 15734
I20260925 00:43:53.843134 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 15865
I20260925 00:43:53.848429 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 15996
I20260925 00:43:53.853786 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 15643
2026-09-25T00:43:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (896 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T00:43:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:53Z Disabled control of system clock
I20260925 00:43:53.883417 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44627
--webserver_interface=127.6.72.126
--webserver_port=0
--builtin_ntp_servers=127.6.72.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:44627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 00:43:53.995992 16135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:53.996310 16135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:53.996349 16135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:54.000640 16135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:54.000741 16135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:54.000757 16135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:54.000774 16135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:54.000790 16135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:54.006691 16135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:44627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:54.007890 16135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:54.009405 16135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:54.018006 16141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:54.018006 16140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:54.018018 16143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:54.018344 16135 server_base.cc:1034] running on GCE node
I20260925 00:43:54.019133 16135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:54.020407 16135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:54.021644 16135 hybrid_clock.cc:648] HybridClock initialized: now 1790297034021615 us; error 31 us; skew 500 ppm
I20260925 00:43:54.023705 16135 webserver.cc:460] Webserver started at http://127.6.72.126:42285/ using document root <none> and password file <none>
I20260925 00:43:54.024376 16135 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:54.024472 16135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:54.024725 16135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:54.026862 16135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "e37f478aee644c5381bd1a6e78882698"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "60a848b9b6dfc94859ad531402c5435b"
server_key_iv: "a5a8a63a6e336c1bbb3e17c367954ef2"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.027462 16135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "e37f478aee644c5381bd1a6e78882698"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "60a848b9b6dfc94859ad531402c5435b"
server_key_iv: "a5a8a63a6e336c1bbb3e17c367954ef2"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.031510 16135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:43:54.034327 16149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:54.035676 16135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:43:54.035843 16135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "e37f478aee644c5381bd1a6e78882698"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "60a848b9b6dfc94859ad531402c5435b"
server_key_iv: "a5a8a63a6e336c1bbb3e17c367954ef2"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.035997 16135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:54.040874 16135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:54.041532 16135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:54.041735 16135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:54.048887 16135 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:44627
I20260925 00:43:54.048906 16201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:44627 every 8 connection(s)
I20260925 00:43:54.049911 16135 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:54.053331 16202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:54.059257 16202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698: Bootstrap starting.
I20260925 00:43:54.059600 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 16135
I20260925 00:43:54.059712 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:54.060006 6433 external_mini_cluster.cc:1428] Setting key 4a8262939cf5e3627387793e28ef6971
I20260925 00:43:54.062289 16202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:54.063063 16202 log.cc:824] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:54.065485 16202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698: No bootstrap required, opened a new log
I20260925 00:43:54.066298 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:54.061490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57810 (local address 127.6.72.126:44627)
0925 00:43:54.062244 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:54.062260 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:54.062297 (+ 37us) server_negotiation.cc:407] Connection header received
0925 00:43:54.063046 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:54.063065 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:54.063535 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:54.063935 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:54.064857 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:54.064862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:54.064874 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:54.064884 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:54.064942 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:54.064959 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:54.065075 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:54.065221 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 00:43:54.065504 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":143,"threads_started":1}
I20260925 00:43:54.069135 16202 raft_consensus.cc:348] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37f478aee644c5381bd1a6e78882698" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44627 } }
I20260925 00:43:54.069360 16202 raft_consensus.cc:374] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:54.069438 16202 raft_consensus.cc:729] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e37f478aee644c5381bd1a6e78882698, State: Initialized, Role: FOLLOWER
I20260925 00:43:54.069885 16202 consensus_queue.cc:260] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37f478aee644c5381bd1a6e78882698" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44627 } }
I20260925 00:43:54.070009 16202 raft_consensus.cc:388] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:54.070096 16202 raft_consensus.cc:482] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:54.070194 16202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:54.071628 16202 raft_consensus.cc:504] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37f478aee644c5381bd1a6e78882698" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44627 } }
I20260925 00:43:54.071983 16202 leader_election.cc:302] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e37f478aee644c5381bd1a6e78882698; no voters:
I20260925 00:43:54.072381 16202 leader_election.cc:288] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:54.072482 16207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:54.072754 16207 raft_consensus.cc:686] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [term 1 LEADER]: Becoming Leader. State: Replica: e37f478aee644c5381bd1a6e78882698, State: Running, Role: LEADER
I20260925 00:43:54.073048 16207 consensus_queue.cc:237] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37f478aee644c5381bd1a6e78882698" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44627 } }
I20260925 00:43:54.073678 16202 sys_catalog.cc:564] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:54.074764 16208 sys_catalog.cc:455] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e37f478aee644c5381bd1a6e78882698" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37f478aee644c5381bd1a6e78882698" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44627 } } }
I20260925 00:43:54.074898 16208 sys_catalog.cc:458] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:54.075207 16209 sys_catalog.cc:455] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e37f478aee644c5381bd1a6e78882698. Latest consensus state: current_term: 1 leader_uuid: "e37f478aee644c5381bd1a6e78882698" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e37f478aee644c5381bd1a6e78882698" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44627 } } }
I20260925 00:43:54.075301 16209 sys_catalog.cc:458] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698 [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:54.075395 16216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:54.078776 16216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:54.084769 16216 catalog_manager.cc:1347] Generated new cluster ID: 54469f87d7e24e1da156984ce27ece2b
I20260925 00:43:54.084858 16216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:54.106276 16216 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:54.106647 16216 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:54.121243 16216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e37f478aee644c5381bd1a6e78882698: Generated new TSK 0
I20260925 00:43:54.122037 16216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:43:54.136343 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44627
--builtin_ntp_servers=127.6.72.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 00:43:54.250618 16226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:54.250898 16226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:54.250941 16226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:54.255125 16226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:54.255195 16226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:54.255254 16226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:54.260085 16226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:54.261247 16226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:54.262683 16226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:54.272584 16232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:54.272668 16226 server_base.cc:1034] running on GCE node
W20260925 00:43:54.272588 16234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:54.272886 16231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:54.273276 16226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:54.273990 16226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:54.275193 16226 hybrid_clock.cc:648] HybridClock initialized: now 1790297034275172 us; error 32 us; skew 500 ppm
I20260925 00:43:54.277118 16226 webserver.cc:460] Webserver started at http://127.6.72.65:44973/ using document root <none> and password file <none>
I20260925 00:43:54.277706 16226 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:54.277768 16226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:54.277997 16226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:54.280057 16226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "24eea724d4fc425f8ef802542bf47806"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "b79f443a7be1402f0fefe0d72a72022e"
server_key_iv: "7cd186c55efc6aad32ddd4f0b117e44b"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.280735 16226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "24eea724d4fc425f8ef802542bf47806"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "b79f443a7be1402f0fefe0d72a72022e"
server_key_iv: "7cd186c55efc6aad32ddd4f0b117e44b"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.284412 16226 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:43:54.287045 16240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:54.288393 16226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:43:54.288520 16226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "24eea724d4fc425f8ef802542bf47806"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "b79f443a7be1402f0fefe0d72a72022e"
server_key_iv: "7cd186c55efc6aad32ddd4f0b117e44b"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.288653 16226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:54.293354 16226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:54.293985 16226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:54.294175 16226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:54.294731 16226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:54.295656 16226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:54.295727 16226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:54.295797 16226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:54.295851 16226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:54.304550 16226 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:45917
I20260925 00:43:54.304586 16353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:45917 every 8 connection(s)
I20260925 00:43:54.305495 16226 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:54.311980 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:54.307644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:52343 (local address 127.6.72.126:44627)
0925 00:43:54.307792 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:54.307797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:54.308877 (+ 1080us) server_negotiation.cc:407] Connection header received
0925 00:43:54.310180 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:54.310183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:54.310221 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:54.310295 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:54.311463 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:54.311472 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:54.311474 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:54.311490 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:54.311511 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:54.311519 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:54.311609 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:54.311820 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 00:43:54.311867 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 00:43:54.312824 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 16226
I20260925 00:43:54.312916 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:54.313071 16356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:54.308042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44627 (local address 127.6.72.65:52343)
0925 00:43:54.308726 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:54.308765 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:54.309956 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:54.310430 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:54.310438 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:54.310528 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:54.311135 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:54.311137 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:54.311195 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:54.311267 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:54.311630 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:54.311638 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:54.311644 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 00:43:54.311815 (+ 171us) client_negotiation.cc:240] Negotiation successful
0925 00:43:54.312105 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":156,"threads_started":1}
I20260925 00:43:54.313265 6433 external_mini_cluster.cc:1428] Setting key 9db56e1051cb6a0525c5cafd00582804
I20260925 00:43:54.314532 16354 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44627
I20260925 00:43:54.314790 16354 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:54.315233 16354 heartbeater.cc:511] Master 127.6.72.126:44627 requested a full tablet report, sending...
I20260925 00:43:54.315456 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44627
--builtin_ntp_servers=127.6.72.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 00:43:54.317001 16165 ts_manager.cc:194] Registered new tserver with Master: 24eea724d4fc425f8ef802542bf47806 (127.6.72.65:45917)
W20260925 00:43:54.427598 16357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:54.427870 16357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:54.427910 16357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:54.432076 16357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:54.432137 16357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:54.432196 16357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:54.437572 16357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:54.438660 16357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:54.440145 16357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:54.449995 16362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:54.450177 16363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:54.450192 16357 server_base.cc:1034] running on GCE node
W20260925 00:43:54.450214 16365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:54.450847 16357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:54.451471 16357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:54.452653 16357 hybrid_clock.cc:648] HybridClock initialized: now 1790297034452634 us; error 30 us; skew 500 ppm
I20260925 00:43:54.454802 16357 webserver.cc:460] Webserver started at http://127.6.72.66:38161/ using document root <none> and password file <none>
I20260925 00:43:54.455443 16357 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:54.455513 16357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:54.455734 16357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:54.457978 16357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "45497de9b2b84edab67f7fc82784a9cb"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "683b3e8f4218450f66e2b346b0b3ab80"
server_key_iv: "2de700698f903493a4f28a1de171d48a"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.458593 16357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "45497de9b2b84edab67f7fc82784a9cb"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "683b3e8f4218450f66e2b346b0b3ab80"
server_key_iv: "2de700698f903493a4f28a1de171d48a"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.462709 16357 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:43:54.465554 16371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:54.466929 16357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 00:43:54.467089 16357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "45497de9b2b84edab67f7fc82784a9cb"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "683b3e8f4218450f66e2b346b0b3ab80"
server_key_iv: "2de700698f903493a4f28a1de171d48a"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.467219 16357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:54.471953 16357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:54.472630 16357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:54.472810 16357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:54.473409 16357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:54.474399 16357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:54.474483 16357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:54.474538 16357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:54.474578 16357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:54.483942 16357 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:33915
I20260925 00:43:54.483979 16484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:33915 every 8 connection(s)
I20260925 00:43:54.484934 16357 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:54.491137 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:54.486887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:35699 (local address 127.6.72.126:44627)
0925 00:43:54.487063 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:54.487067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:54.488072 (+ 1005us) server_negotiation.cc:407] Connection header received
0925 00:43:54.489461 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:54.489465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:54.489505 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:54.489584 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:54.490709 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:54.490713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:54.490715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:54.490728 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:54.490749 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:54.490757 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:54.490842 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:54.490933 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 00:43:54.491026 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:43:54.492154 16487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:54.487250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44627 (local address 127.6.72.66:35699)
0925 00:43:54.487903 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:54.487944 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:54.489236 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:54.489744 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:54.489751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:54.489868 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:54.490440 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:54.490441 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:54.490485 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:54.490554 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:54.490836 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:54.490842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:54.490846 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 00:43:54.491007 (+ 161us) client_negotiation.cc:240] Negotiation successful
0925 00:43:54.491230 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":107,"threads_started":1}
I20260925 00:43:54.492228 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 16357
I20260925 00:43:54.492410 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:54.492785 6433 external_mini_cluster.cc:1428] Setting key 421114a568326f254cc8996c9a9981aa
I20260925 00:43:54.493450 16485 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44627
I20260925 00:43:54.493693 16485 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:54.494110 16485 heartbeater.cc:511] Master 127.6.72.126:44627 requested a full tablet report, sending...
I20260925 00:43:54.494741 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44627
--builtin_ntp_servers=127.6.72.84:35861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 00:43:54.495312 16165 ts_manager.cc:194] Registered new tserver with Master: 45497de9b2b84edab67f7fc82784a9cb (127.6.72.66:33915)
W20260925 00:43:54.608006 16488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:54.608325 16488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:54.608367 16488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:54.612362 16488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:54.612426 16488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:54.612493 16488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:54.620213 16488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:54.621822 16488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:54.623510 16488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:54.633541 16493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:54.633514 16496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:54.633517 16494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:54.633607 16488 server_base.cc:1034] running on GCE node
I20260925 00:43:54.634189 16488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:54.634879 16488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:54.636114 16488 hybrid_clock.cc:648] HybridClock initialized: now 1790297034636086 us; error 45 us; skew 500 ppm
I20260925 00:43:54.638195 16488 webserver.cc:460] Webserver started at http://127.6.72.67:33361/ using document root <none> and password file <none>
I20260925 00:43:54.638810 16488 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:54.638906 16488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:54.639144 16488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:54.641480 16488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "93b2707d0b93452da665b9ebb80a788b"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "1b1df6cde65c8f999580dd16a03c3566"
server_key_iv: "0d23c299f98f28bfce6f336781d60e03"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.642050 16488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "93b2707d0b93452da665b9ebb80a788b"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "1b1df6cde65c8f999580dd16a03c3566"
server_key_iv: "0d23c299f98f28bfce6f336781d60e03"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.645841 16488 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:43:54.648631 16502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:54.649991 16488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:54.650161 16488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "93b2707d0b93452da665b9ebb80a788b"
format_stamp: "Formatted at 2026-09-25 00:43:54 on dist-test-slave-w8nh"
server_key: "1b1df6cde65c8f999580dd16a03c3566"
server_key_iv: "0d23c299f98f28bfce6f336781d60e03"
server_key_version: "encryptionkey@0"
I20260925 00:43:54.650295 16488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:54.655144 16488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:54.655778 16488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:54.655963 16488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:54.656575 16488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:54.657483 16488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:54.657560 16488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:54.657626 16488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:54.657672 16488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:54.666393 16488 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:43539
I20260925 00:43:54.666419 16615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:43539 every 8 connection(s)
I20260925 00:43:54.667301 16488 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:54.671495 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 16488
I20260925 00:43:54.671645 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:54.671944 6433 external_mini_cluster.cc:1428] Setting key 3137dce7cc76a5b3bfaaf73c8a161f4c
I20260925 00:43:54.673902 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:54.669377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:47791 (local address 127.6.72.126:44627)
0925 00:43:54.669521 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:54.669530 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:54.670628 (+ 1098us) server_negotiation.cc:407] Connection header received
0925 00:43:54.672073 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:54.672077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:54.672127 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:54.672202 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:54.673380 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:54.673383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:54.673386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:43:54.673398 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:54.673420 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:43:54.673428 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:54.673538 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:54.673738 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 00:43:54.673781 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:43:54.674875 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:54.669728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44627 (local address 127.6.72.67:47791)
0925 00:43:54.670475 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:54.670521 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:54.671839 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:54.672424 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:54.672434 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:54.672549 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:54.673131 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:43:54.673132 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:54.673172 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:43:54.673239 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:54.673567 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:54.673576 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:54.673582 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 00:43:54.673749 (+ 167us) client_negotiation.cc:240] Negotiation successful
0925 00:43:54.673941 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":114,"threads_started":1}
I20260925 00:43:54.676159 16616 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44627
I20260925 00:43:54.676440 16616 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:54.676828 16616 heartbeater.cc:511] Master 127.6.72.126:44627 requested a full tablet report, sending...
I20260925 00:43:54.677856 16165 ts_manager.cc:194] Registered new tserver with Master: 93b2707d0b93452da665b9ebb80a788b (127.6.72.67:43539)
I20260925 00:43:54.686303 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 00:43:54.689208 16205 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:43:54.688311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57814 (local address 127.6.72.126:44627)
0925 00:43:54.688462 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:54.688469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:54.688600 (+ 131us) server_negotiation.cc:407] Connection header received
0925 00:43:54.688705 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:54.688707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:54.688742 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:54.688806 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:54.689045 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57814: BlockingRecv error: recv got EOF from 127.0.0.1:57814 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:43:54.689294 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 16226
I20260925 00:43:54.694654 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 16357
I20260925 00:43:54.700075 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 16488
I20260925 00:43:54.705312 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 16135
2026-09-25T00:43:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (851 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:54 dist-test-slave-w8nh krb5kdc[16626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:54 dist-test-slave-w8nh krb5kdc[16626](info): set up 2 sockets
Sep 25 00:43:54 dist-test-slave-w8nh krb5kdc[16626](info): commencing operation
krb5kdc: starting...
W20260925 00:43:55.753407 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:55 dist-test-slave-w8nh krb5kdc[16626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:43:55.883948 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:34763
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:34763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:55.997697 16639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:55.997967 16639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:55.998039 16639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:56.002254 16639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:43:56.002324 16639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:56.002344 16639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:56.002362 16639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:43:56.002411 16639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:43:56.007954 16639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:34763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:34763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:56.009152 16639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:56.010275 16639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:56.019651 16644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:56.019644 16645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:56.019665 16647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:56.020295 16639 server_base.cc:1034] running on GCE node
I20260925 00:43:56.020918 16639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:56.022046 16639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:56.023228 16639 hybrid_clock.cc:648] HybridClock initialized: now 1790297036023208 us; error 34 us; skew 500 ppm
Sep 25 00:43:56 dist-test-slave-w8nh krb5kdc[16626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297036, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:56.032696 16639 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:43:56.034013 16639 webserver.cc:460] Webserver started at http://127.6.72.126:45937/ using document root <none> and password file <none>
I20260925 00:43:56.034605 16639 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:56.034695 16639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:56.034909 16639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:56.037084 16639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "221a56959efa4a8cae3f1d24eacad21c"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "7f0a1392b5bf0439efa7905010051e93"
server_key_iv: "bf32d73a724adc0160262dcff422a802"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.037654 16639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "221a56959efa4a8cae3f1d24eacad21c"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "7f0a1392b5bf0439efa7905010051e93"
server_key_iv: "bf32d73a724adc0160262dcff422a802"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.041471 16639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260925 00:43:56.043881 16654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:56.045189 16639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:43:56.045354 16639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "221a56959efa4a8cae3f1d24eacad21c"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "7f0a1392b5bf0439efa7905010051e93"
server_key_iv: "bf32d73a724adc0160262dcff422a802"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.045482 16639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:56.066344 16639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:56.070056 16639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:56.070307 16639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:56.077543 16639 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:34763
I20260925 00:43:56.077912 16706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:34763 every 8 connection(s)
I20260925 00:43:56.078590 16639 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:43:56.080900 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 16639
I20260925 00:43:56.081032 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:43:56.081317 6433 external_mini_cluster.cc:1428] Setting key 552039b89f952e13c58dba7a3a2f34b9
I20260925 00:43:56.081645 16707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:56.089154 16707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c: Bootstrap starting.
Sep 25 00:43:56 dist-test-slave-w8nh krb5kdc[16626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:56.091883 16707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:56.092842 16707 log.cc:824] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:56.094739 16707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c: No bootstrap required, opened a new log
I20260925 00:43:56.096534 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.082768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56184 (local address 127.6.72.126:34763)
0925 00:43:56.083190 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.083210 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.083252 (+ 42us) server_negotiation.cc:407] Connection header received
0925 00:43:56.084011 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.084035 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.084503 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.084903 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:56.085528 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.087140 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.088020 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.088764 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.088939 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.091654 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:56.091659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:56.091672 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:56.091699 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:56.093500 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:56.093943 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:56.093953 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:56.093958 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:56.094094 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:56.094394 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:56.094401 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:56.094402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:56.094806 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:56.095004 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.095305 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.095585 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":107,"threads_started":1}
I20260925 00:43:56.098065 16707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a56959efa4a8cae3f1d24eacad21c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34763 } }
I20260925 00:43:56.098340 16707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:56.098405 16707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 221a56959efa4a8cae3f1d24eacad21c, State: Initialized, Role: FOLLOWER
I20260925 00:43:56.098865 16707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a56959efa4a8cae3f1d24eacad21c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34763 } }
I20260925 00:43:56.099026 16707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:43:56.099076 16707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:43:56.099144 16707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:56.100061 16707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a56959efa4a8cae3f1d24eacad21c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34763 } }
I20260925 00:43:56.100425 16707 leader_election.cc:302] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 221a56959efa4a8cae3f1d24eacad21c; no voters:
I20260925 00:43:56.100724 16707 leader_election.cc:288] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:43:56.100896 16712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:56.101154 16712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [term 1 LEADER]: Becoming Leader. State: Replica: 221a56959efa4a8cae3f1d24eacad21c, State: Running, Role: LEADER
I20260925 00:43:56.101511 16712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a56959efa4a8cae3f1d24eacad21c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34763 } }
I20260925 00:43:56.102062 16707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:43:56.103294 16714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 221a56959efa4a8cae3f1d24eacad21c. Latest consensus state: current_term: 1 leader_uuid: "221a56959efa4a8cae3f1d24eacad21c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a56959efa4a8cae3f1d24eacad21c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34763 } } }
I20260925 00:43:56.103464 16714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:56.103294 16713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "221a56959efa4a8cae3f1d24eacad21c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a56959efa4a8cae3f1d24eacad21c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34763 } } }
I20260925 00:43:56.103575 16713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c [sys.catalog]: This master's current role is: LEADER
I20260925 00:43:56.103902 16722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:43:56.106583 16722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:43:56.112039 16722 catalog_manager.cc:1347] Generated new cluster ID: 840dac87cbdc4d8ebbbb0c1f3798ce69
I20260925 00:43:56.112124 16722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:43:56.120138 16722 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:43:56.121181 16722 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:43:56.126406 16722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c: Generated new TSK 0
I20260925 00:43:56.127126 16722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:43:56.164710 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34763
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:56.285156 16733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:56.285455 16733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:56.285501 16733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:56.289757 16733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:56.289832 16733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:56.289927 16733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:43:56.295509 16733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:56.296730 16733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:56.297796 16733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:56.307874 16738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:56.307878 16739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:56.307874 16741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:56.308271 16733 server_base.cc:1034] running on GCE node
I20260925 00:43:56.308696 16733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:56.309401 16733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:56.310600 16733 hybrid_clock.cc:648] HybridClock initialized: now 1790297036310582 us; error 33 us; skew 500 ppm
Sep 25 00:43:56 dist-test-slave-w8nh krb5kdc[16626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297036, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:56.313661 16733 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:43:56.314574 16733 webserver.cc:460] Webserver started at http://127.6.72.65:43517/ using document root <none> and password file <none>
I20260925 00:43:56.315162 16733 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:56.315255 16733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:56.315491 16733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:56.317615 16733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "31fa7acd8e9d484d98453c0e56c1c38e"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "98d68c9d6e975fa2e1bd66247364c3fe"
server_key_iv: "8bfd136096e6089ee91615f34007d60a"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.318189 16733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "31fa7acd8e9d484d98453c0e56c1c38e"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "98d68c9d6e975fa2e1bd66247364c3fe"
server_key_iv: "8bfd136096e6089ee91615f34007d60a"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.322208 16733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 00:43:56.324836 16748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:56.326278 16733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:56.326426 16733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "31fa7acd8e9d484d98453c0e56c1c38e"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "98d68c9d6e975fa2e1bd66247364c3fe"
server_key_iv: "8bfd136096e6089ee91615f34007d60a"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.326560 16733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:56.339095 16733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:56.342485 16733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:56.342705 16733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:56.343321 16733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:56.344159 16733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:56.344230 16733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:56.344347 16733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:56.344398 16733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:56.353993 16733 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:35955
I20260925 00:43:56.354004 16861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:35955 every 8 connection(s)
I20260925 00:43:56.354936 16733 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:43:56.362316 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 16733
I20260925 00:43:56.362473 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:43:56.362762 6433 external_mini_cluster.cc:1428] Setting key b2fca6b744bd7588cb974c0e594ee9d4
Sep 25 00:43:56 dist-test-slave-w8nh krb5kdc[16626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297036, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:56.369366 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.356806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:33633 (local address 127.6.72.126:34763)
0925 00:43:56.356959 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.356963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.357907 (+ 944us) server_negotiation.cc:407] Connection header received
0925 00:43:56.359287 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.359290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.359350 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.359459 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:56.360723 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.361260 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.362408 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.362748 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.362855 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.365602 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:56.365610 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:56.365612 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:56.365636 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:56.367304 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:56.367798 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:56.367804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:56.367805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:56.367893 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:56.368226 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:56.368231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:56.368232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:56.368444 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:56.368554 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.369060 (+ 506us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.369191 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:43:56.370183 16864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.357205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:34763 (local address 127.6.72.65:33633)
0925 00:43:56.357759 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:56.357802 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:56.359070 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:56.359647 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:56.359654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:56.360011 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:56.360565 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.360579 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.361387 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.361390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.362279 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.362288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.362908 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.362913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.363082 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.363782 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:56.363802 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:56.365429 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:56.367413 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:56.367418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:56.367434 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:56.367696 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:56.367983 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:56.367987 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:56.367990 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:56.368135 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:56.368551 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:56.368558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:56.368816 (+ 258us) client_negotiation.cc:769] Sending connection context
0925 00:43:56.369001 (+ 185us) client_negotiation.cc:240] Negotiation successful
0925 00:43:56.369333 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":116,"threads_started":1}
I20260925 00:43:56.371620 16862 heartbeater.cc:347] Connected to a master server at 127.6.72.126:34763
I20260925 00:43:56.371863 16862 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:56.372403 16862 heartbeater.cc:511] Master 127.6.72.126:34763 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:43:56.374008 16671 ts_manager.cc:194] Registered new tserver with Master: 31fa7acd8e9d484d98453c0e56c1c38e (127.6.72.65:35955)
I20260925 00:43:56.375420 16671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:33633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:43:56.391813 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34763
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:56.505405 16867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:56.505671 16867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:56.505716 16867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:56.509817 16867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:56.509884 16867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:56.509948 16867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:43:56.515640 16867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:56.516839 16867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:56.517899 16867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:56.527859 16872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:56.527875 16875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:56.527904 16873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:56.528152 16867 server_base.cc:1034] running on GCE node
I20260925 00:43:56.528669 16867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:56.529390 16867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:56.530645 16867 hybrid_clock.cc:648] HybridClock initialized: now 1790297036530599 us; error 59 us; skew 500 ppm
Sep 25 00:43:56 dist-test-slave-w8nh krb5kdc[16626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297036, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:56.534564 16867 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:43:56.535655 16867 webserver.cc:460] Webserver started at http://127.6.72.66:43145/ using document root <none> and password file <none>
I20260925 00:43:56.536329 16867 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:56.536422 16867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:56.536650 16867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:56.538830 16867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "d947edcce3a44fce9ba8021ac50f5f52"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "5bffe1c9a9d6892ec1e01d759e0e8aca"
server_key_iv: "c88038e19994685039b6354997796f15"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.539415 16867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "d947edcce3a44fce9ba8021ac50f5f52"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "5bffe1c9a9d6892ec1e01d759e0e8aca"
server_key_iv: "c88038e19994685039b6354997796f15"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.543320 16867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:43:56.545886 16882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:56.547147 16867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:43:56.547269 16867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "d947edcce3a44fce9ba8021ac50f5f52"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "5bffe1c9a9d6892ec1e01d759e0e8aca"
server_key_iv: "c88038e19994685039b6354997796f15"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.547377 16867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:56.565779 16867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:56.569229 16867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:56.569443 16867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:56.570055 16867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:56.570899 16867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:56.570951 16867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:56.571022 16867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:56.571053 16867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:56.580315 16867 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:39391
I20260925 00:43:56.580333 16995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:39391 every 8 connection(s)
I20260925 00:43:56.581297 16867 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:43:56.589257 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 16867
I20260925 00:43:56.589421 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:43:56.589818 6433 external_mini_cluster.cc:1428] Setting key 71d5cbe383fca304ebca375fb424a0e0
Sep 25 00:43:56 dist-test-slave-w8nh krb5kdc[16626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297036, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:56.599252 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.583242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:32917 (local address 127.6.72.126:34763)
0925 00:43:56.583457 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.583462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.584524 (+ 1062us) server_negotiation.cc:407] Connection header received
0925 00:43:56.586398 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.586403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.586486 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.586595 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:56.588064 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.588827 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.590976 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.591249 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.591393 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.594550 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:56.594553 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:56.594555 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:56.594581 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:56.596176 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:56.597072 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:56.597075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:56.597076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:56.597158 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:56.597716 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:56.597719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:56.597720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:56.597929 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:56.598049 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.598993 (+ 944us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.599116 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 00:43:56.600806 16998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.583678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:34763 (local address 127.6.72.66:32917)
0925 00:43:56.584312 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:56.584386 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:56.586115 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:56.586807 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:56.586816 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:56.587286 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:56.587863 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.587878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.589060 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.589064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.590772 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.590783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.591446 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.591451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.591672 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.592421 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:56.592445 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:56.594338 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:56.596355 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:56.596365 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:56.596390 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:56.596914 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:56.597278 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:56.597285 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:56.597290 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:56.597568 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:56.598079 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:56.598092 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:56.598570 (+ 478us) client_negotiation.cc:769] Sending connection context
0925 00:43:56.598862 (+ 292us) client_negotiation.cc:240] Negotiation successful
0925 00:43:56.599333 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":140,"threads_started":1}
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:43:56.602273 16996 heartbeater.cc:347] Connected to a master server at 127.6.72.126:34763
I20260925 00:43:56.602757 16996 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:56.603591 16996 heartbeater.cc:511] Master 127.6.72.126:34763 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:43:56.605209 16671 ts_manager.cc:194] Registered new tserver with Master: d947edcce3a44fce9ba8021ac50f5f52 (127.6.72.66:39391)
I20260925 00:43:56.606001 16671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:32917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:43:56.624725 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34763
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:43:56.739743 17001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:43:56.740103 17001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:43:56.740190 17001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:43:56.744522 17001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:43:56.744633 17001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:43:56.744725 17001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:43:56.750571 17001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:43:56.751835 17001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:43:56.753002 17001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:43:56.763151 17009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:56.763166 17007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:43:56.763151 17006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:43:56.763638 17001 server_base.cc:1034] running on GCE node
I20260925 00:43:56.764063 17001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:43:56.764792 17001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:43:56.765990 17001 hybrid_clock.cc:648] HybridClock initialized: now 1790297036765974 us; error 34 us; skew 500 ppm
Sep 25 00:43:56 dist-test-slave-w8nh krb5kdc[16626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297036, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:43:56.768884 17001 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:43:56.769908 17001 webserver.cc:460] Webserver started at http://127.6.72.67:46049/ using document root <none> and password file <none>
I20260925 00:43:56.770493 17001 fs_manager.cc:359] Metadata directory not provided
I20260925 00:43:56.770604 17001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:43:56.770849 17001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:43:56.773059 17001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "e00fd8b148b54f308266d6ac7eccc58f"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "978ce233750693a8350ac22b24d0d8ed"
server_key_iv: "585c34b281c9584eb0d33e30b401d16c"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.773654 17001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "e00fd8b148b54f308266d6ac7eccc58f"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "978ce233750693a8350ac22b24d0d8ed"
server_key_iv: "585c34b281c9584eb0d33e30b401d16c"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.777567 17001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:43:56.780386 17016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:56.781754 17001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:43:56.781898 17001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "e00fd8b148b54f308266d6ac7eccc58f"
format_stamp: "Formatted at 2026-09-25 00:43:56 on dist-test-slave-w8nh"
server_key: "978ce233750693a8350ac22b24d0d8ed"
server_key_iv: "585c34b281c9584eb0d33e30b401d16c"
server_key_version: "encryptionkey@0"
I20260925 00:43:56.782020 17001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:43:56.800272 17001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:43:56.803897 17001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:43:56.804123 17001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:43:56.804808 17001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:43:56.805686 17001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:43:56.805756 17001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:56.805822 17001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:43:56.805866 17001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:43:56.814754 17001 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:33213
I20260925 00:43:56.814777 17129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:33213 every 8 connection(s)
I20260925 00:43:56.815682 17001 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:43:56.822389 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 17001
I20260925 00:43:56.822501 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:43:56.822782 6433 external_mini_cluster.cc:1428] Setting key bda6c8195f2cb9821f20e8010efaf2c7
Sep 25 00:43:56 dist-test-slave-w8nh krb5kdc[16626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297036, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:43:56.830192 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.817734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:48519 (local address 127.6.72.126:34763)
0925 00:43:56.817882 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.817886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.818752 (+ 866us) server_negotiation.cc:407] Connection header received
0925 00:43:56.820153 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.820157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.820219 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.820353 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:56.821554 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.822088 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.823777 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.824015 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.824140 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.826751 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:56.826754 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:56.826756 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:56.826780 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:56.828019 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:56.828633 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:56.828637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:56.828639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:56.828720 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:56.829068 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:56.829071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:56.829073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:56.829264 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:56.829390 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.829894 (+ 504us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.830018 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:43:56.831151 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.818141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:34763 (local address 127.6.72.67:48519)
0925 00:43:56.818604 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:56.818647 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:56.819910 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:56.820528 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:56.820535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:56.820911 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:43:56.821395 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.821408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.822264 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.822267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.823651 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.823660 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.824180 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.824185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.824390 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.825048 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:43:56.825071 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:43:56.826580 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:43:56.828172 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:56.828178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:56.828194 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:56.828522 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:56.828818 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:43:56.828821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:43:56.828824 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:43:56.828965 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:43:56.829388 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:43:56.829395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:43:56.829666 (+ 271us) client_negotiation.cc:769] Sending connection context
0925 00:43:56.829855 (+ 189us) client_negotiation.cc:240] Negotiation successful
0925 00:43:56.830196 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":110,"threads_started":1}
I20260925 00:43:56.832365 17130 heartbeater.cc:347] Connected to a master server at 127.6.72.126:34763
I20260925 00:43:56.832618 17130 heartbeater.cc:464] Registering TS with master...
I20260925 00:43:56.833144 17130 heartbeater.cc:511] Master 127.6.72.126:34763 requested a full tablet report, sending...
I20260925 00:43:56.834223 16671 ts_manager.cc:194] Registered new tserver with Master: e00fd8b148b54f308266d6ac7eccc58f (127.6.72.67:33213)
I20260925 00:43:56.834764 16671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:48519
I20260925 00:43:56.837007 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:43:56.846099 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.838892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.6.72.126:34763)
0925 00:43:56.839032 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.839073 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.839196 (+ 123us) server_negotiation.cc:407] Connection header received
0925 00:43:56.839288 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.839290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.839329 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.839396 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:43:56.840057 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.840714 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.841582 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.841785 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.841861 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.843247 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:43:56.843250 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:43:56.843252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:43:56.843270 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:43:56.844571 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:56.844955 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:56.844958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:56.844959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:56.845027 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:43:56.845279 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:43:56.845281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:43:56.845282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:43:56.845481 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:43:56.845580 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.845777 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.845864 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 00:43:56.849498 16671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:43:56.852847 16671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:43:56.868342 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.862448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:39391 (local address 127.0.0.1:56730)
0925 00:43:56.863210 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:56.863267 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:56.863396 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:56.864072 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:56.864076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:56.864101 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:56.864233 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.864239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.865475 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.865482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.866496 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.866504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.867819 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.867822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.867895 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.867922 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 00:43:56.868047 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 00:43:56.868169 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":99,"threads_started":1}
I20260925 00:43:56.868355 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.862594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.6.72.66:39391)
0925 00:43:56.862933 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.862948 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.863338 (+ 390us) server_negotiation.cc:407] Connection header received
0925 00:43:56.863587 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.863593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.863794 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.863894 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:56.864387 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.865342 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.866661 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.867494 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.867627 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.867758 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.868085 (+ 327us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.868178 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":99,"threads_started":1}
I20260925 00:43:56.868642 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.863197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33213 (local address 127.0.0.1:39232)
0925 00:43:56.863495 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:56.863510 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:56.863593 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:56.864437 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:56.864440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:56.864459 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:56.864576 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.864591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.866150 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.866154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.867231 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.867243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.868227 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.868231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.868340 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.868365 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 00:43:56.868417 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 00:43:56.868495 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":26,"thread_start_us":99,"threads_started":1}
I20260925 00:43:56.869294 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.863608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39232 (local address 127.6.72.67:33213)
0925 00:43:56.863959 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.863970 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.864001 (+ 31us) server_negotiation.cc:407] Connection header received
0925 00:43:56.864072 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.864079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.864285 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.864709 (+ 424us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:56.864863 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.866006 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.867368 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.868090 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.868830 (+ 740us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.868986 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.869037 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.869123 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":169,"threads_started":1}
I20260925 00:43:56.870469 17146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.864043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.6.72.65:35955)
0925 00:43:56.864402 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.864418 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.864440 (+ 22us) server_negotiation.cc:407] Connection header received
0925 00:43:56.864507 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.864512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.864688 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.864831 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:56.865072 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.866162 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.867619 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.868939 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.869048 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.869166 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.869279 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.870114 (+ 835us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":116,"threads_started":1}
I20260925 00:43:56.871456 16930 tablet_service.cc:1467] Processing CreateTablet for tablet 6f8cf4f2ba434a65b90bf3069e44b897 (DEFAULT_TABLE table=test-table [id=634030fb6f184f438fb6e75e81cecb18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:56.871555 17064 tablet_service.cc:1467] Processing CreateTablet for tablet 6f8cf4f2ba434a65b90bf3069e44b897 (DEFAULT_TABLE table=test-table [id=634030fb6f184f438fb6e75e81cecb18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:56.872448 17144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.862823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35955 (local address 127.0.0.1:39790)
0925 00:43:56.863474 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:56.863488 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:56.863578 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:56.864832 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:56.864835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:56.864853 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:56.864940 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.864946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.866300 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.866304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.867500 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.867509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.869058 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.869062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.869143 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.869165 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:43:56.872191 (+ 3026us) client_negotiation.cc:240] Negotiation successful
0925 00:43:56.872310 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":29,"thread_start_us":99,"threads_started":1}
I20260925 00:43:56.872481 16930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f8cf4f2ba434a65b90bf3069e44b897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:56.872503 17064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f8cf4f2ba434a65b90bf3069e44b897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:56.875300 16796 tablet_service.cc:1467] Processing CreateTablet for tablet 6f8cf4f2ba434a65b90bf3069e44b897 (DEFAULT_TABLE table=test-table [id=634030fb6f184f438fb6e75e81cecb18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:43:56.876474 16796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f8cf4f2ba434a65b90bf3069e44b897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:43:56.879736 17147 tablet_bootstrap.cc:492] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: Bootstrap starting.
I20260925 00:43:56.880090 17148 tablet_bootstrap.cc:492] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: Bootstrap starting.
I20260925 00:43:56.881719 17147 tablet_bootstrap.cc:654] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:56.882685 17148 tablet_bootstrap.cc:654] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:56.882862 17147 log.cc:824] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:56.883504 17148 log.cc:824] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:56.883565 17150 tablet_bootstrap.cc:492] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e: Bootstrap starting.
I20260925 00:43:56.884948 17147 tablet_bootstrap.cc:492] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: No bootstrap required, opened a new log
I20260925 00:43:56.885162 17147 ts_tablet_manager.cc:1397] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:43:56.886011 17148 tablet_bootstrap.cc:492] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: No bootstrap required, opened a new log
I20260925 00:43:56.886215 17148 ts_tablet_manager.cc:1397] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260925 00:43:56.886657 17150 tablet_bootstrap.cc:654] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e: Neither blocks nor log segments found. Creating new log.
I20260925 00:43:56.887465 17150 log.cc:824] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e: Log is configured to *not* fsync() on all Append() calls
I20260925 00:43:56.889115 17147 raft_consensus.cc:348] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d947edcce3a44fce9ba8021ac50f5f52" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39391 } } peers { permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 } } peers { permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 } }
I20260925 00:43:56.889393 17147 raft_consensus.cc:374] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:56.889452 17147 raft_consensus.cc:729] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00fd8b148b54f308266d6ac7eccc58f, State: Initialized, Role: FOLLOWER
I20260925 00:43:56.889490 17150 tablet_bootstrap.cc:492] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e: No bootstrap required, opened a new log
I20260925 00:43:56.889668 17150 ts_tablet_manager.cc:1397] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:43:56.889494 17148 raft_consensus.cc:348] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d947edcce3a44fce9ba8021ac50f5f52" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39391 } } peers { permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 } } peers { permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 } }
I20260925 00:43:56.889778 17148 raft_consensus.cc:374] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:56.889837 17148 raft_consensus.cc:729] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d947edcce3a44fce9ba8021ac50f5f52, State: Initialized, Role: FOLLOWER
I20260925 00:43:56.889925 17147 consensus_queue.cc:260] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d947edcce3a44fce9ba8021ac50f5f52" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39391 } } peers { permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 } } peers { permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 } }
I20260925 00:43:56.890264 17148 consensus_queue.cc:260] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d947edcce3a44fce9ba8021ac50f5f52" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39391 } } peers { permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 } } peers { permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 } }
I20260925 00:43:56.890663 17130 heartbeater.cc:503] Master 127.6.72.126:34763 was elected leader, sending a full tablet report...
I20260925 00:43:56.891119 17148 ts_tablet_manager.cc:1428] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 00:43:56.891484 16996 heartbeater.cc:503] Master 127.6.72.126:34763 was elected leader, sending a full tablet report...
I20260925 00:43:56.893121 17147 ts_tablet_manager.cc:1428] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260925 00:43:56.894052 17150 raft_consensus.cc:348] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d947edcce3a44fce9ba8021ac50f5f52" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39391 } } peers { permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 } } peers { permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 } }
I20260925 00:43:56.894318 17150 raft_consensus.cc:374] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:43:56.894379 17150 raft_consensus.cc:729] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31fa7acd8e9d484d98453c0e56c1c38e, State: Initialized, Role: FOLLOWER
I20260925 00:43:56.894831 17150 consensus_queue.cc:260] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d947edcce3a44fce9ba8021ac50f5f52" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39391 } } peers { permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 } } peers { permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 } }
I20260925 00:43:56.895454 16862 heartbeater.cc:503] Master 127.6.72.126:34763 was elected leader, sending a full tablet report...
I20260925 00:43:56.895787 17150 ts_tablet_manager.cc:1428] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 00:43:56.960412 17154 raft_consensus.cc:482] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:43:56.960625 17154 raft_consensus.cc:504] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d947edcce3a44fce9ba8021ac50f5f52" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39391 } } peers { permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 } } peers { permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 } }
I20260925 00:43:56.961782 17154 leader_election.cc:288] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e00fd8b148b54f308266d6ac7eccc58f (127.6.72.67:33213), 31fa7acd8e9d484d98453c0e56c1c38e (127.6.72.65:35955)
I20260925 00:43:56.965399 16998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.961999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33213 (local address 127.6.72.66:42313)
0925 00:43:56.962120 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:56.962137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:56.962230 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:56.962526 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:56.962529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:56.962557 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:56.962681 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.962686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.963467 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.963470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.964287 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.964296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.964947 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.964951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.965056 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.965080 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 00:43:56.965188 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 00:43:56.965266 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"spinlock_wait_cycles":16128}
I20260925 00:43:56.965399 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.962109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:42313 (local address 127.6.72.67:33213)
0925 00:43:56.962250 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.962257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.962276 (+ 19us) server_negotiation.cc:407] Connection header received
0925 00:43:56.962333 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.962336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.962395 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.962499 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:56.962800 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.963327 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.964412 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.964815 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.964931 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.964996 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.965204 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.965275 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:43:56.965601 17146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.962174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:35499 (local address 127.6.72.65:35955)
0925 00:43:56.962300 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.962306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.962559 (+ 253us) server_negotiation.cc:407] Connection header received
0925 00:43:56.962710 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.962713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.962762 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.962845 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:43:56.963096 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.963585 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.964654 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.965071 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.965187 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.965249 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.965423 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.965493 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 00:43:56.965615 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.962005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35955 (local address 127.6.72.66:35499)
0925 00:43:56.962462 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:56.962481 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 00:43:56.962604 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:43:56.962856 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:43:56.962859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:43:56.962874 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:43:56.962989 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.962994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.963698 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.963702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.964539 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:43:56.964548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.965190 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.965193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:43:56.965302 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.965320 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 00:43:56.965407 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 00:43:56.965480 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":115,"thread_start_us":133,"threads_started":1}
I20260925 00:43:56.966135 17084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f8cf4f2ba434a65b90bf3069e44b897" candidate_uuid: "d947edcce3a44fce9ba8021ac50f5f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00fd8b148b54f308266d6ac7eccc58f" is_pre_election: true
I20260925 00:43:56.966274 16816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f8cf4f2ba434a65b90bf3069e44b897" candidate_uuid: "d947edcce3a44fce9ba8021ac50f5f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" is_pre_election: true
I20260925 00:43:56.966485 17084 raft_consensus.cc:2454] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d947edcce3a44fce9ba8021ac50f5f52 in term 0.
I20260925 00:43:56.966574 16816 raft_consensus.cc:2454] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d947edcce3a44fce9ba8021ac50f5f52 in term 0.
I20260925 00:43:56.967032 16886 leader_election.cc:302] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 31fa7acd8e9d484d98453c0e56c1c38e, d947edcce3a44fce9ba8021ac50f5f52, e00fd8b148b54f308266d6ac7eccc58f; no voters:
I20260925 00:43:56.967286 17154 raft_consensus.cc:2781] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:43:56.967376 17154 raft_consensus.cc:482] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:43:56.967437 17154 raft_consensus.cc:3037] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:56.968616 17154 raft_consensus.cc:504] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d947edcce3a44fce9ba8021ac50f5f52" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39391 } } peers { permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 } } peers { permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 } }
I20260925 00:43:56.969012 17154 leader_election.cc:288] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [CANDIDATE]: Term 1 election: Requested vote from peers e00fd8b148b54f308266d6ac7eccc58f (127.6.72.67:33213), 31fa7acd8e9d484d98453c0e56c1c38e (127.6.72.65:35955)
I20260925 00:43:56.969369 16816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f8cf4f2ba434a65b90bf3069e44b897" candidate_uuid: "d947edcce3a44fce9ba8021ac50f5f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31fa7acd8e9d484d98453c0e56c1c38e"
I20260925 00:43:56.969369 17084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f8cf4f2ba434a65b90bf3069e44b897" candidate_uuid: "d947edcce3a44fce9ba8021ac50f5f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e00fd8b148b54f308266d6ac7eccc58f"
I20260925 00:43:56.969528 16816 raft_consensus.cc:3037] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:56.969527 17084 raft_consensus.cc:3037] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:43:56.970678 17084 raft_consensus.cc:2454] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d947edcce3a44fce9ba8021ac50f5f52 in term 1.
I20260925 00:43:56.970992 16886 leader_election.cc:302] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d947edcce3a44fce9ba8021ac50f5f52, e00fd8b148b54f308266d6ac7eccc58f; no voters:
I20260925 00:43:56.971113 16816 raft_consensus.cc:2454] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d947edcce3a44fce9ba8021ac50f5f52 in term 1.
I20260925 00:43:56.971189 17154 raft_consensus.cc:2781] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:43:56.971485 17154 raft_consensus.cc:686] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 1 LEADER]: Becoming Leader. State: Replica: d947edcce3a44fce9ba8021ac50f5f52, State: Running, Role: LEADER
I20260925 00:43:56.971848 17154 consensus_queue.cc:237] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d947edcce3a44fce9ba8021ac50f5f52" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39391 } } peers { permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 } } peers { permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 } }
I20260925 00:43:56.975366 16670 catalog_manager.cc:5495] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 reported cstate change: term changed from 0 to 1, leader changed from <none> to d947edcce3a44fce9ba8021ac50f5f52 (127.6.72.66). New cstate: current_term: 1 leader_uuid: "d947edcce3a44fce9ba8021ac50f5f52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d947edcce3a44fce9ba8021ac50f5f52" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 } health_report { overall_health: UNKNOWN } } }
I20260925 00:43:56.993290 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:56.989544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.6.72.66:39391)
0925 00:43:56.989667 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 00:43:56.989670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:43:56.989686 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:43:56.989773 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:43:56.989776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:43:56.989834 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:43:56.989997 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:43:56.990300 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.990806 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.991713 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:43:56.991939 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:43:56.992068 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:43:56.992477 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:43:56.992922 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:43:56.993053 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 00:43:56.993112 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 00:43:56.993174 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 00:43:56.999599 16816 raft_consensus.cc:1261] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e [term 1 FOLLOWER]: Refusing update from remote peer d947edcce3a44fce9ba8021ac50f5f52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:56.999599 17084 raft_consensus.cc:1261] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f [term 1 FOLLOWER]: Refusing update from remote peer d947edcce3a44fce9ba8021ac50f5f52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:43:57.000394 17157 consensus_queue.cc:1035] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e00fd8b148b54f308266d6ac7eccc58f" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:57.000641 17154 consensus_queue.cc:1035] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31fa7acd8e9d484d98453c0e56c1c38e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:43:57.019024 16671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:57.019217 16671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:43:57.021816 16671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 221a56959efa4a8cae3f1d24eacad21c: Sending DeleteTablet for 3 replicas of tablet 6f8cf4f2ba434a65b90bf3069e44b897
I20260925 00:43:57.022768 16796 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f8cf4f2ba434a65b90bf3069e44b897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:57 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:39790
I20260925 00:43:57.022753 16930 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f8cf4f2ba434a65b90bf3069e44b897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:57 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:56730
I20260925 00:43:57.022940 17064 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f8cf4f2ba434a65b90bf3069e44b897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:43:57 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:39232
I20260925 00:43:57.023322 17170 tablet_replica.cc:330] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e: stopping tablet replica
I20260925 00:43:57.023665 17171 tablet_replica.cc:330] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: stopping tablet replica
I20260925 00:43:57.023727 17170 raft_consensus.cc:2229] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:57.023788 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 16733
I20260925 00:43:57.024073 17170 raft_consensus.cc:2258] T 6f8cf4f2ba434a65b90bf3069e44b897 P 31fa7acd8e9d484d98453c0e56c1c38e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:57.024442 17169 tablet_replica.cc:330] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: stopping tablet replica
I20260925 00:43:57.025039 17171 raft_consensus.cc:2229] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:43:57.025071 17169 raft_consensus.cc:2229] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:43:57.025337 17171 raft_consensus.cc:2258] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:57.025450 17169 raft_consensus.cc:2258] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:43:57.026988 17169 ts_tablet_manager.cc:1905] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:57.026988 17171 ts_tablet_manager.cc:1905] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:43:57.029768 17171 ts_tablet_manager.cc:1918] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:57.029768 17169 ts_tablet_manager.cc:1918] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:43:57.029876 17171 log.cc:1196] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/6f8cf4f2ba434a65b90bf3069e44b897
I20260925 00:43:57.029875 17169 log.cc:1196] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/6f8cf4f2ba434a65b90bf3069e44b897
I20260925 00:43:57.030231 17171 ts_tablet_manager.cc:1939] T 6f8cf4f2ba434a65b90bf3069e44b897 P e00fd8b148b54f308266d6ac7eccc58f: Deleting consensus metadata
I20260925 00:43:57.030231 17169 ts_tablet_manager.cc:1939] T 6f8cf4f2ba434a65b90bf3069e44b897 P d947edcce3a44fce9ba8021ac50f5f52: Deleting consensus metadata
I20260925 00:43:57.031474 16658 catalog_manager.cc:4841] TS d947edcce3a44fce9ba8021ac50f5f52 (127.6.72.66:39391): tablet 6f8cf4f2ba434a65b90bf3069e44b897 (table test-table [id=634030fb6f184f438fb6e75e81cecb18]) successfully deleted
I20260925 00:43:57.031733 16658 catalog_manager.cc:4841] TS e00fd8b148b54f308266d6ac7eccc58f (127.6.72.67:33213): tablet 6f8cf4f2ba434a65b90bf3069e44b897 (table test-table [id=634030fb6f184f438fb6e75e81cecb18]) successfully deleted
W20260925 00:43:57.034204 16656 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:35955 (error 108)
I20260925 00:43:57.035080 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 16867
I20260925 00:43:57.035398 17144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:43:57.035093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35955 (local address 127.0.0.1:39804)
0925 00:43:57.035248 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 00:43:57.035313 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:35955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260925 00:43:57.036654 16656 catalog_manager.cc:4584] TS 31fa7acd8e9d484d98453c0e56c1c38e (127.6.72.65:35955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f8cf4f2ba434a65b90bf3069e44b897: Network error: Client connection negotiation failed: client connection to 127.6.72.65:35955: connect: Connection refused (error 111)
I20260925 00:43:57.041477 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 17001
I20260925 00:43:57.047765 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 16639
2026-09-25T00:43:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2345 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:43:57 dist-test-slave-w8nh krb5kdc[17175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:43:57 dist-test-slave-w8nh krb5kdc[17175](info): set up 2 sockets
Sep 25 00:43:57 dist-test-slave-w8nh krb5kdc[17175](info): commencing operation
krb5kdc: starting...
W20260925 00:43:58.093281 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:43:58 dist-test-slave-w8nh krb5kdc[17175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:43:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:43:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.72.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.72.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/rangeradmin_127.6.72.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/rangeradmin_127.6.72.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.72.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.72.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/rangerlookup_127.6.72.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/rangerlookup_127.6.72.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.72.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.72.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/HTTP_127.6.72.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/HTTP_127.6.72.84@KRBTEST.COM.keytab.
I20260925 00:43:58.271023 6433 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasklhk22w/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/postgres -l logfile start
2026-09-25 00:44:00.905 UTC [17205] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 00:44:00.905 UTC [17205] LOG: listening on IPv4 address "127.6.72.84", port 32987
2026-09-25 00:44:00.909 UTC [17205] LOG: listening on Unix socket "/tmp/.s.PGSQL.32987"
2026-09-25 00:44:00.931 UTC [17208] LOG: database system was shut down at 2026-09-25 00:43:59 UTC
2026-09-25 00:44:00.935 UTC [17205] LOG: database system is ready to accept connections
I20260925 00:44:01.893400 6433 mini_postgres.cc:96] Postgres bound to 32987
2026-09-25 00:44:01.899 UTC [17216] FATAL: database "slave" does not exist
127.6.72.84:32987 - accepting connections
I20260925 00:44:01.900741 6433 mini_ranger.cc:161] Starting Ranger...
I20260925 00:44:01.916348 6433 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 00:44:02.447336 6433 mini_ranger.cc:87] Created ranger Postgres database
I20260925 00:44:02.447461 6433 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-admin
2026-09-25 00:44:02,732 [I] DB FLAVOR :POSTGRES
2026-09-25 00:44:02,732 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 00:44:02,732 [I] Checking connection..
2026-09-25 00:44:02,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 00:44:02,979 [I] Checking connection passed.
2026-09-25 00:44:02,979 [I] --------- Verifying version history table ---------
2026-09-25 00:44:02,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 00:44:03,240 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 00:44:03,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 00:44:03,484 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 00:44:03,484 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 00:44:03,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 00:44:03.751 UTC [17280] WARNING: there is no transaction in progress
2026-09-25 00:44:03,757 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 00:44:03,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 00:44:04,024 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 00:44:04,024 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 00:44:04,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 00:44:04,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 00:44:04,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-w8nh.c.gcp-upstream.internal','N') ;"
2026-09-25 00:44:04,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 00:44:05,054 [I] Table x_portal_user does not exist in database ranger
2026-09-25 00:44:05,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 00:44:05,321 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 00:44:05,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 00:44:05,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 00:44:05,821 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 00:44:05,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 00:44:06.869 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:06.888 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:06.907 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:06.924 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:06.941 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:06.964 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07.037 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07.044 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07.060 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07.072 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07.084 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07.100 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07.111 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07.119 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07.126 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07.135 UTC [17388] WARNING: there is no transaction in progress
2026-09-25 00:44:07,892 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 00:44:07,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 00:44:08,149 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 00:44:08,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 00:44:08,408 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 00:44:08,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 00:44:08,655 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 00:44:08,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 00:44:08,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 00:44:09,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-w8nh.c.gcp-upstream.internal';"
2026-09-25 00:44:09,408 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 00:44:09,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 00:44:09,649 [I] DB_PATCHES have already been applied
I20260925 00:44:09.656319 6433 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-admin:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasklhk22w/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 00:44:09.656420 6433 mini_ranger.cc:193] Using host: 127.6.72.84
I20260925 00:44:09.660365 6433 mini_ranger.cc:227] Ranger admin URL: http://127.6.72.84:51345
Sep 25, 2026 12:44:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 12:44:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 12:44:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 12:44:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 12:44:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 12:44:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 12:44:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 00:44:10.785998 6433 mini_ranger.cc:160] Time spent starting Ranger: real 8.885s user 0.003s sys 0.008s
Sep 25, 2026 12:44:31 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 12:44:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 12:44:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 12:44:31 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 12:44:32 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 00:44:32.840454 6433 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.72.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.72.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/rangerkms_127.6.72.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/rangerkms_127.6.72.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.72.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.72.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.72.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/HTTP_127.6.72.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.72.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/HTTP_127.6.72.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 00:44:32 dist-test-slave-w8nh krb5kdc[17175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297072, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 00:44:32.933324 6433 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 00:44:32.944300 6433 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 00:44:33.849709 6433 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 00:44:33.849807 6433 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-kms
I20260925 00:44:33.849910 6433 mini_ranger_kms.cc:227] Using postgres at 127.6.72.84:32987
2026-09-25 00:44:34,072 [I] DB FLAVOR :POSTGRES
2026-09-25 00:44:34,072 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 00:44:34,072 [I] Checking connection
2026-09-25 00:44:34,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 00:44:34,370 [I] connection success
2026-09-25 00:44:34,370 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 00:44:34,370 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 00:44:34,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 00:44:34,641 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 00:44:34,642 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 00:44:34,642 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 00:44:34,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:32987/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 00:44:34,943 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 00:44:35.441313 6433 mini_ranger_kms.cc:310] Created kms service
I20260925 00:44:35.672138 6433 mini_ranger_kms.cc:326] Created kudu user
I20260925 00:44:35.717592 6433 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 00:44:35 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297072, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:44:36.147003 6433 mini_ranger_kms.cc:384] Added ranger policy
I20260925 00:44:36.147166 6433 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-kms:/tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 00:44:36.147187 6433 mini_ranger_kms.cc:240] Using host: 127.6.72.84
I20260925 00:44:36.151228 6433 mini_ranger_kms.cc:276] Ranger KMS PID: 17568
I20260925 00:44:36.151388 6433 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.72.84:34215
Sep 25, 2026 12:44:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 12:44:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 12:44:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 12:44:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 12:44:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 12:44:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 00:44:38.478830 6433 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.546s user 0.024s sys 0.179s
I20260925 00:44:38.478978 6433 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 00:44:38.479115 6433 mini_ranger_kms.cc:401] 127.6.72.84:34215/kms/v1/keys
Sep 25 00:44:38 dist-test-slave-w8nh krb5kdc[17175](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790297078, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.72.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 00:44:39 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297078, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.72.84@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
Sep 25 00:44:39 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297078, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.72.84@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
Sep 25 00:44:41 dist-test-slave-w8nh krb5kdc[17175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:44:41.779901 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:46173
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:46173
--ranger_config_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:34215/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:44:41.895252 17619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:44:41.895565 17619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:44:41.895655 17619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:44:41.899744 17619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:44:41.899852 17619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:44:41.899896 17619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:44:41.899955 17619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:44:41.900008 17619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:44:41.905745 17619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.72.84:34215/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:46173
--ranger_config_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:46173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:44:41.906909 17619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:44:41.907970 17619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:44:41.918215 17624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:44:41.918283 17627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:44:41.918268 17625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:44:41.918572 17619 server_base.cc:1034] running on GCE node
I20260925 00:44:41.919243 17619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:44:41.920454 17619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:44:41.921654 17619 hybrid_clock.cc:648] HybridClock initialized: now 1790297081921632 us; error 40 us; skew 500 ppm
Sep 25 00:44:41 dist-test-slave-w8nh krb5kdc[17175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297081, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:44:41.924788 17619 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:44:41.925954 17619 webserver.cc:460] Webserver started at http://127.6.72.126:42415/ using document root <none> and password file <none>
I20260925 00:44:41.926641 17619 fs_manager.cc:359] Metadata directory not provided
I20260925 00:44:41.926728 17619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:44:41.926962 17619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 00:44:41 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297081, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:44:41.975013 17619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "07d6f7fd0ca948f7b26db231a19ebe04"
format_stamp: "Formatted at 2026-09-25 00:44:41 on dist-test-slave-w8nh"
server_key: "51a36995e60766e26849583ad411a405"
server_key_iv: "89e3c4ae938dde130ab7b629978f8bb1"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:41.975708 17619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "07d6f7fd0ca948f7b26db231a19ebe04"
format_stamp: "Formatted at 2026-09-25 00:44:41 on dist-test-slave-w8nh"
server_key: "51a36995e60766e26849583ad411a405"
server_key_iv: "89e3c4ae938dde130ab7b629978f8bb1"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:41.979733 17619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:44:41.997341 17634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:44:41.998709 17619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:44:41.998850 17619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "07d6f7fd0ca948f7b26db231a19ebe04"
format_stamp: "Formatted at 2026-09-25 00:44:41 on dist-test-slave-w8nh"
server_key: "51a36995e60766e26849583ad411a405"
server_key_iv: "89e3c4ae938dde130ab7b629978f8bb1"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:41.998986 17619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:44:42.016646 17619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:44:42.019521 17619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:44:42.019714 17619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:44:42.027853 17619 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:46173
I20260925 00:44:42.027902 17686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:46173 every 8 connection(s)
I20260925 00:44:42.028895 17619 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:44:42.032120 17687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:44:42.037802 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 17619
I20260925 00:44:42.037941 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:44:42.038228 6433 external_mini_cluster.cc:1428] Setting key 7b8943bfcc2d4cc842637210fe3b8e2f
I20260925 00:44:42.039127 17687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04: Bootstrap starting.
I20260925 00:44:42.041885 17687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04: Neither blocks nor log segments found. Creating new log.
I20260925 00:44:42.042825 17687 log.cc:824] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04: Log is configured to *not* fsync() on all Append() calls
I20260925 00:44:42.044926 17687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04: No bootstrap required, opened a new log
Sep 25 00:44:42 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:44:42.049512 17687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d6f7fd0ca948f7b26db231a19ebe04" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46173 } }
I20260925 00:44:42.049831 17687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:44:42.049926 17687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07d6f7fd0ca948f7b26db231a19ebe04, State: Initialized, Role: FOLLOWER
I20260925 00:44:42.050444 17687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d6f7fd0ca948f7b26db231a19ebe04" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46173 } }
I20260925 00:44:42.050596 17687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:44:42.050678 17687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:44:42.050800 17687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:44:42.051987 17687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d6f7fd0ca948f7b26db231a19ebe04" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46173 } }
I20260925 00:44:42.052402 17687 leader_election.cc:302] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07d6f7fd0ca948f7b26db231a19ebe04; no voters:
I20260925 00:44:42.052759 17687 leader_election.cc:288] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:44:42.052989 17692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:44:42.053349 17692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [term 1 LEADER]: Becoming Leader. State: Replica: 07d6f7fd0ca948f7b26db231a19ebe04, State: Running, Role: LEADER
I20260925 00:44:42.053763 17692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d6f7fd0ca948f7b26db231a19ebe04" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46173 } }
I20260925 00:44:42.054132 17687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:44:42.054456 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:42.039700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45300 (local address 127.6.72.126:46173)
0925 00:44:42.040082 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:42.040105 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:42.040139 (+ 34us) server_negotiation.cc:407] Connection header received
0925 00:44:42.040825 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:42.040841 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:42.041172 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:42.041514 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:44:42.042318 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:42.043479 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:42.044768 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:42.045549 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:42.045703 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:42.048925 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:44:42.048933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:44:42.048952 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:44:42.048984 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:44:42.051091 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:44:42.051577 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:44:42.051583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:44:42.051589 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:44:42.051719 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:44:42.052060 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:44:42.052063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:44:42.052065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:44:42.052622 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:44:42.052840 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:42.053216 (+ 376us) server_negotiation.cc:299] Negotiation successful
0925 00:44:42.053503 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":122,"threads_started":1}
I20260925 00:44:42.055440 17692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07d6f7fd0ca948f7b26db231a19ebe04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d6f7fd0ca948f7b26db231a19ebe04" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46173 } } }
I20260925 00:44:42.055634 17692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [sys.catalog]: This master's current role is: LEADER
I20260925 00:44:42.055868 17692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07d6f7fd0ca948f7b26db231a19ebe04. Latest consensus state: current_term: 1 leader_uuid: "07d6f7fd0ca948f7b26db231a19ebe04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d6f7fd0ca948f7b26db231a19ebe04" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46173 } } }
I20260925 00:44:42.055994 17692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04 [sys.catalog]: This master's current role is: LEADER
I20260925 00:44:42.056313 17701 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:44:42.059559 17701 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:44:42.060038 17687 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 00:44:42.078137 17701 catalog_manager.cc:1347] Generated new cluster ID: 1703b30b0cba4252a9925dbd48ec74a1
I20260925 00:44:42.078220 17701 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:44:42.107854 17701 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:44:42.109205 17701 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:44:42.116291 17701 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04: Generated new TSK 0
I20260925 00:44:42.117175 17701 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:44:42.849646 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:34215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46173
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:44:42.999760 17732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:44:43.000159 17732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:44:43.000223 17732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:44:43.006876 17732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:44:43.006966 17732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:44:43.007062 17732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:44:43.015383 17732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.72.84:34215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:44:43.016840 17732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:44:43.018160 17732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:44:43.031556 17742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:44:43.031931 17739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:44:43.032193 17740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:44:43.033408 17732 server_base.cc:1034] running on GCE node
I20260925 00:44:43.034003 17732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:44:43.034838 17732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:44:43.036019 17732 hybrid_clock.cc:648] HybridClock initialized: now 1790297083035983 us; error 48 us; skew 500 ppm
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:44:43.039590 17732 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:44:43.040660 17732 webserver.cc:460] Webserver started at http://127.6.72.65:37003/ using document root <none> and password file <none>
I20260925 00:44:43.041384 17732 fs_manager.cc:359] Metadata directory not provided
I20260925 00:44:43.041468 17732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:44:43.041672 17732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:44:43.060611 17732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "d5fc50dd4f524498a0cd1c42af878b7d"
format_stamp: "Formatted at 2026-09-25 00:44:43 on dist-test-slave-w8nh"
server_key: "16b64a380de880c6a447603a496f550d"
server_key_iv: "4e1cec73b9952b04ed6a1d48ea37046b"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:43.061510 17732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "d5fc50dd4f524498a0cd1c42af878b7d"
format_stamp: "Formatted at 2026-09-25 00:44:43 on dist-test-slave-w8nh"
server_key: "16b64a380de880c6a447603a496f550d"
server_key_iv: "4e1cec73b9952b04ed6a1d48ea37046b"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:43.066891 17732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 00:44:43.083838 17749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:44:43.085520 17732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:44:43.085696 17732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "d5fc50dd4f524498a0cd1c42af878b7d"
format_stamp: "Formatted at 2026-09-25 00:44:43 on dist-test-slave-w8nh"
server_key: "16b64a380de880c6a447603a496f550d"
server_key_iv: "4e1cec73b9952b04ed6a1d48ea37046b"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:43.085877 17732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:44:43.099685 17732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:44:43.103384 17732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:44:43.103628 17732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:44:43.104331 17732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:44:43.105383 17732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:44:43.105489 17732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:44:43.105611 17732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:44:43.105690 17732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:44:43.119467 17732 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:35397
I20260925 00:44:43.120537 17732 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:44:43.121126 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 17732
I20260925 00:44:43.121259 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:44:43.121636 6433 external_mini_cluster.cc:1428] Setting key 3c9c601227c2aaec8e6d4a1063457f27
I20260925 00:44:43.121649 17863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:35397 every 8 connection(s)
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:44:43.171700 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:43.125551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:46083 (local address 127.6.72.126:46173)
0925 00:44:43.125917 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:43.125922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:43.129007 (+ 3085us) server_negotiation.cc:407] Connection header received
0925 00:44:43.141932 (+ 12925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:43.141936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:43.142014 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:43.142108 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:44:43.150858 (+ 8750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.151763 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.154075 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.156194 (+ 2119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.156319 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:43.166718 (+ 10399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:44:43.166723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:44:43.166726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:44:43.166772 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:44:43.168977 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:44:43.169721 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:44:43.169725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:44:43.169727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:44:43.169837 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:44:43.170272 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:44:43.170275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:44:43.170277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:44:43.170522 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:44:43.170608 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:43.171365 (+ 757us) server_negotiation.cc:299] Negotiation successful
0925 00:44:43.171506 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":91,"threads_started":1}
I20260925 00:44:43.173641 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:43.124435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:46173 (local address 127.6.72.65:46083)
0925 00:44:43.128757 (+ 4322us) negotiation.cc:107] Waiting for socket to connect
0925 00:44:43.128854 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0925 00:44:43.141263 (+ 12409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:44:43.149372 (+ 8109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:44:43.149385 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:44:43.149811 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:44:43.150614 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:43.150638 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.151945 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.151948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:43.153872 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:43.153933 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.156560 (+ 2627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.156564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:43.156797 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:43.164513 (+ 7716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:44:43.164541 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:44:43.166439 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:44:43.169163 (+ 2724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:44:43.169171 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:44:43.169191 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:44:43.169590 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:44:43.169952 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:44:43.169955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:44:43.169957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:44:43.170158 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:44:43.170676 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:44:43.170683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:44:43.171068 (+ 385us) client_negotiation.cc:769] Sending connection context
0925 00:44:43.172204 (+ 1136us) client_negotiation.cc:240] Negotiation successful
0925 00:44:43.172589 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3967,"thread_start_us":137,"threads_started":1}
I20260925 00:44:43.175364 17864 heartbeater.cc:347] Connected to a master server at 127.6.72.126:46173
I20260925 00:44:43.175606 17864 heartbeater.cc:464] Registering TS with master...
I20260925 00:44:43.176419 17864 heartbeater.cc:511] Master 127.6.72.126:46173 requested a full tablet report, sending...
I20260925 00:44:43.179283 17651 ts_manager.cc:194] Registered new tserver with Master: d5fc50dd4f524498a0cd1c42af878b7d (127.6.72.65:35397)
I20260925 00:44:43.181598 17651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:46083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:44:43.199958 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:34215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46173
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 00:44:43.364360 17871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:44:43.364806 17871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:44:43.364924 17871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:44:43.370808 17871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:44:43.370981 17871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:44:43.371124 17871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:44:43.378841 17871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.72.84:34215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:44:43.380501 17871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:44:43.381845 17871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:44:43.400769 17878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:44:43.400771 17880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:44:43.402181 17877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:44:43.402298 17871 server_base.cc:1034] running on GCE node
I20260925 00:44:43.403010 17871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:44:43.403838 17871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:44:43.405125 17871 hybrid_clock.cc:648] HybridClock initialized: now 1790297083405057 us; error 86 us; skew 500 ppm
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:44:43.408674 17871 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:44:43.409749 17871 webserver.cc:460] Webserver started at http://127.6.72.66:43033/ using document root <none> and password file <none>
I20260925 00:44:43.410501 17871 fs_manager.cc:359] Metadata directory not provided
I20260925 00:44:43.410585 17871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:44:43.410840 17871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:44:43.429288 17871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "9d3b8d94c60b465bb9b0ca0882e70810"
format_stamp: "Formatted at 2026-09-25 00:44:43 on dist-test-slave-w8nh"
server_key: "e309f558866a69f6acaf1c959e454729"
server_key_iv: "b0473e59e319efa43264e7b32040e8ac"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:43.430059 17871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "9d3b8d94c60b465bb9b0ca0882e70810"
format_stamp: "Formatted at 2026-09-25 00:44:43 on dist-test-slave-w8nh"
server_key: "e309f558866a69f6acaf1c959e454729"
server_key_iv: "b0473e59e319efa43264e7b32040e8ac"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:43.435714 17871 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 00:44:43.464502 17887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:44:43.466219 17871 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:44:43.466391 17871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "9d3b8d94c60b465bb9b0ca0882e70810"
format_stamp: "Formatted at 2026-09-25 00:44:43 on dist-test-slave-w8nh"
server_key: "e309f558866a69f6acaf1c959e454729"
server_key_iv: "b0473e59e319efa43264e7b32040e8ac"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:43.466560 17871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:44:43.491811 17871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:44:43.495988 17871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:44:43.496199 17871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:44:43.496999 17871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:44:43.498087 17871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:44:43.498155 17871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:44:43.498214 17871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:44:43.498247 17871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:44:43.510488 17871 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:43813
I20260925 00:44:43.512272 17871 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:44:43.517011 18000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:43813 every 8 connection(s)
I20260925 00:44:43.522301 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 17871
I20260925 00:44:43.522454 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:44:43.522802 6433 external_mini_cluster.cc:1428] Setting key c923df72ac4043dc868536bfb46f6d03
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:44:43.550623 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:43.520858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:51027 (local address 127.6.72.126:46173)
0925 00:44:43.521006 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:43.521010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:43.521866 (+ 856us) server_negotiation.cc:407] Connection header received
0925 00:44:43.527470 (+ 5604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:43.527475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:43.527545 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:43.527604 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:44:43.531228 (+ 3624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.532093 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.538995 (+ 6902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.539382 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.539481 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:43.545700 (+ 6219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:44:43.545705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:44:43.545708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:44:43.545744 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:44:43.547739 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:44:43.548463 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:44:43.548467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:44:43.548469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:44:43.548561 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:44:43.548982 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:44:43.548985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:44:43.548987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:44:43.549213 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:44:43.549294 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:43.550274 (+ 980us) server_negotiation.cc:299] Negotiation successful
0925 00:44:43.550442 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:44:43.552386 18003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:43.519527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:46173 (local address 127.6.72.66:51027)
0925 00:44:43.521666 (+ 2139us) negotiation.cc:107] Waiting for socket to connect
0925 00:44:43.521727 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 00:44:43.527161 (+ 5434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:44:43.528108 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:44:43.528119 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:44:43.528535 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:44:43.529297 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:43.529320 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.532313 (+ 2993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.532316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:43.533791 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:43.533804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.539734 (+ 5930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.539739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:43.539993 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:43.540780 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:44:43.540806 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:44:43.545452 (+ 4646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:44:43.547895 (+ 2443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:44:43.547902 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:44:43.547923 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:44:43.548323 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:44:43.548672 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:44:43.548675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:44:43.548677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:44:43.548868 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:44:43.549362 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:44:43.549368 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:44:43.549961 (+ 593us) client_negotiation.cc:769] Sending connection context
0925 00:44:43.550880 (+ 919us) client_negotiation.cc:240] Negotiation successful
0925 00:44:43.551258 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1905,"thread_start_us":190,"threads_started":1}
I20260925 00:44:43.553960 18001 heartbeater.cc:347] Connected to a master server at 127.6.72.126:46173
I20260925 00:44:43.554226 18001 heartbeater.cc:464] Registering TS with master...
I20260925 00:44:43.554875 18001 heartbeater.cc:511] Master 127.6.72.126:46173 requested a full tablet report, sending...
I20260925 00:44:43.556107 17651 ts_manager.cc:194] Registered new tserver with Master: 9d3b8d94c60b465bb9b0ca0882e70810 (127.6.72.66:43813)
I20260925 00:44:43.556970 17651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:51027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:44:43.572160 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:34215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46173
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:44:43.743873 18006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:44:43.744340 18006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:44:43.744427 18006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:44:43.751164 18006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:44:43.751260 18006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:44:43.751379 18006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:44:43.760562 18006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.72.84:34215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:44:43.762320 18006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:44:43.763860 18006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
W20260925 00:44:43.778990 18012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:44:43.779001 18014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:44:43.780380 18006 server_base.cc:1034] running on GCE node
W20260925 00:44:43.781219 18011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:44:43.781802 18006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:44:43.782684 18006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:44:43.790735 18006 hybrid_clock.cc:648] HybridClock initialized: now 1790297083790645 us; error 95 us; skew 500 ppm
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:44:43.794723 18006 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:44:43.795680 18006 webserver.cc:460] Webserver started at http://127.6.72.67:33623/ using document root <none> and password file <none>
I20260925 00:44:43.796505 18006 fs_manager.cc:359] Metadata directory not provided
I20260925 00:44:43.796608 18006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:44:43.796893 18006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:44:43.818907 18006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a"
format_stamp: "Formatted at 2026-09-25 00:44:43 on dist-test-slave-w8nh"
server_key: "726656c743af613cfa8734de1529b8a3"
server_key_iv: "34e3184e7bd0be17af1c3de5efd50559"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:43.819754 18006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a"
format_stamp: "Formatted at 2026-09-25 00:44:43 on dist-test-slave-w8nh"
server_key: "726656c743af613cfa8734de1529b8a3"
server_key_iv: "34e3184e7bd0be17af1c3de5efd50559"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:43.842319 18006 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.008s sys 0.000s
I20260925 00:44:43.871156 18021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:44:43.873919 18006 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.002s sys 0.000s
I20260925 00:44:43.874081 18006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a"
format_stamp: "Formatted at 2026-09-25 00:44:43 on dist-test-slave-w8nh"
server_key: "726656c743af613cfa8734de1529b8a3"
server_key_iv: "34e3184e7bd0be17af1c3de5efd50559"
server_key_version: "kuduclusterkey@0"
I20260925 00:44:43.874238 18006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:44:43.900100 18006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:44:43.903990 18006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:44:43.904273 18006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:44:43.905031 18006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:44:43.906095 18006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:44:43.906208 18006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:44:43.906320 18006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:44:43.906394 18006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:44:43.916775 18006 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:32837
I20260925 00:44:43.917944 18006 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:44:43.918903 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 18006
I20260925 00:44:43.919039 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:44:43.919350 6433 external_mini_cluster.cc:1428] Setting key 584c7ced69854b16d0ad1ef43f039289
I20260925 00:44:43.934111 18134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:32837 every 8 connection(s)
Sep 25 00:44:43 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:44:43.952234 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:43.935776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:55441 (local address 127.6.72.126:46173)
0925 00:44:43.935926 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:43.935930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:43.937794 (+ 1864us) server_negotiation.cc:407] Connection header received
0925 00:44:43.938980 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:43.938984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:43.939037 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:43.939088 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:44:43.940712 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.941567 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.943279 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.943626 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.943731 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:43.947061 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:44:43.947065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:44:43.947068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:44:43.947099 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:44:43.949036 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:44:43.949756 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:44:43.949760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:44:43.949762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:44:43.949865 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:44:43.950395 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:44:43.950399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:44:43.950401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:44:43.950629 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:44:43.950726 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:43.951925 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0925 00:44:43.952063 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:44:43.953382 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:43.936401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:46173 (local address 127.6.72.67:55441)
0925 00:44:43.937635 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0925 00:44:43.937699 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 00:44:43.938723 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:44:43.939486 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:44:43.939495 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:44:43.939845 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:44:43.940542 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:43.940560 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.941722 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.941725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:43.943126 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:43.943137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.943926 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.943931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:43.944140 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:43.944920 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:44:43.944942 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:44:43.946841 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:44:43.949174 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:44:43.949182 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:44:43.949204 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:44:43.949621 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:44:43.949972 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:44:43.949977 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:44:43.949979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:44:43.950208 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:44:43.950754 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:44:43.950763 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:44:43.951593 (+ 830us) client_negotiation.cc:769] Sending connection context
0925 00:44:43.951841 (+ 248us) client_negotiation.cc:240] Negotiation successful
0925 00:44:43.952290 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"thread_start_us":150,"threads_started":1}
I20260925 00:44:43.954975 18135 heartbeater.cc:347] Connected to a master server at 127.6.72.126:46173
I20260925 00:44:43.955225 18135 heartbeater.cc:464] Registering TS with master...
I20260925 00:44:43.955830 18135 heartbeater.cc:511] Master 127.6.72.126:46173 requested a full tablet report, sending...
I20260925 00:44:43.959713 17651 ts_manager.cc:194] Registered new tserver with Master: bb0f5ebe6e3f4a139d86f3a5b4daf61a (127.6.72.67:32837)
I20260925 00:44:43.960628 17651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:55441
I20260925 00:44:43.970906 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:44:43.983424 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:43.973343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45308 (local address 127.6.72.126:46173)
0925 00:44:43.973464 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:43.973468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:43.974044 (+ 576us) server_negotiation.cc:407] Connection header received
0925 00:44:43.974215 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:43.974219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:43.974265 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:43.974310 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:44:43.975223 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.976004 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.977398 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:43.977709 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:43.977780 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:43.979626 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:44:43.979630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:44:43.979633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:44:43.979683 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:44:43.981595 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:44:43.982090 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:44:43.982094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:44:43.982096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:44:43.982184 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:44:43.982475 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:44:43.982479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:44:43.982481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:44:43.982686 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:44:43.982782 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:43.983145 (+ 363us) server_negotiation.cc:299] Negotiation successful
0925 00:44:43.983263 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 00:44:43.987332 17651 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:44:43.991823 17651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:44:44.024098 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.005278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:32837 (local address 127.0.0.1:49982)
0925 00:44:44.009342 (+ 4064us) negotiation.cc:107] Waiting for socket to connect
0925 00:44:44.009356 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:44:44.009431 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:44:44.012374 (+ 2943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:44:44.012381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:44:44.012407 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:44:44.012580 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:44.012587 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.020409 (+ 7822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.020413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:44.021531 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:44.021542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.023533 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.023536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:44.023637 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.023661 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 00:44:44.023775 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 00:44:44.023932 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4001,"thread_start_us":99,"threads_started":1}
I20260925 00:44:44.024797 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.006965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35397 (local address 127.0.0.1:41194)
0925 00:44:44.010559 (+ 3594us) negotiation.cc:107] Waiting for socket to connect
0925 00:44:44.010576 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:44:44.010674 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:44:44.012937 (+ 2263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:44:44.012940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:44:44.012956 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:44:44.013049 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:44.013054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.016274 (+ 3220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.016277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:44.017674 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:44.017685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.020066 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.020069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:44.020184 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.020224 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 00:44:44.024598 (+ 4374us) client_negotiation.cc:240] Negotiation successful
0925 00:44:44.024679 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3511,"thread_start_us":83,"threads_started":1}
I20260925 00:44:44.025993 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.007342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.6.72.65:35397)
0925 00:44:44.011804 (+ 4462us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:44.011812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:44.011841 (+ 29us) server_negotiation.cc:407] Connection header received
0925 00:44:44.011934 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:44.011939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:44.012180 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:44.012785 (+ 605us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:44:44.015041 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.016116 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.018896 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.019729 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.019832 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.020004 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:44.025724 (+ 5720us) server_negotiation.cc:299] Negotiation successful
0925 00:44:44.025853 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3492,"thread_start_us":116,"threads_started":1}
I20260925 00:44:44.026571 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.005685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49982 (local address 127.6.72.67:32837)
0925 00:44:44.009687 (+ 4002us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:44.009698 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:44.009722 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:44:44.009801 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:44.009806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:44.010061 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:44.010252 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:44:44.013188 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.018079 (+ 4891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.021704 (+ 3625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.022533 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.022635 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.026279 (+ 3644us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:44.026336 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 00:44:44.026437 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3886,"thread_start_us":107,"threads_started":1}
I20260925 00:44:44.030254 17791 tablet_service.cc:1467] Processing CreateTablet for tablet 9663738cead849d2906d679ab1568e53 (DEFAULT_TABLE table=test-table [id=be87ab9a1e9b4505bbafcf3cf1d8d56e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:44:44.031438 17791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9663738cead849d2906d679ab1568e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:44:44.074591 18069 tablet_service.cc:1467] Processing CreateTablet for tablet 9663738cead849d2906d679ab1568e53 (DEFAULT_TABLE table=test-table [id=be87ab9a1e9b4505bbafcf3cf1d8d56e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:44:44.076010 18069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9663738cead849d2906d679ab1568e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:44:44.085690 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.003656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:43813 (local address 127.0.0.1:47338)
0925 00:44:44.008044 (+ 4388us) negotiation.cc:107] Waiting for socket to connect
0925 00:44:44.008958 (+ 914us) client_negotiation.cc:174] Beginning negotiation
0925 00:44:44.009097 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:44:44.079077 (+ 69980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:44:44.079081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:44:44.079110 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:44:44.079236 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:44.079243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.081108 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.081113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:44.082284 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:44.082299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.085250 (+ 2951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.085254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:44.085360 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.085390 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 00:44:44.085445 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 00:44:44.085532 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4183,"thread_start_us":99,"threads_started":1}
I20260925 00:44:44.086701 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.003800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47338 (local address 127.6.72.66:43813)
0925 00:44:44.070422 (+ 66622us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:44.070440 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:44.070469 (+ 29us) server_negotiation.cc:407] Connection header received
0925 00:44:44.070556 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:44.070562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:44.070802 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:44.079477 (+ 8675us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:44:44.079632 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.080959 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.082463 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.083258 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.086232 (+ 2974us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.086410 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:44.086467 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 00:44:44.086568 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66492,"thread_start_us":105,"threads_started":1}
I20260925 00:44:44.090976 18155 tablet_bootstrap.cc:492] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: Bootstrap starting.
I20260925 00:44:44.092698 18154 tablet_bootstrap.cc:492] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d: Bootstrap starting.
I20260925 00:44:44.092473 17932 tablet_service.cc:1467] Processing CreateTablet for tablet 9663738cead849d2906d679ab1568e53 (DEFAULT_TABLE table=test-table [id=be87ab9a1e9b4505bbafcf3cf1d8d56e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:44:44.093624 17932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9663738cead849d2906d679ab1568e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:44:44.094563 18155 tablet_bootstrap.cc:654] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: Neither blocks nor log segments found. Creating new log.
I20260925 00:44:44.095470 18154 tablet_bootstrap.cc:654] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d: Neither blocks nor log segments found. Creating new log.
I20260925 00:44:44.095472 18155 log.cc:824] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: Log is configured to *not* fsync() on all Append() calls
I20260925 00:44:44.098621 18154 log.cc:824] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d: Log is configured to *not* fsync() on all Append() calls
I20260925 00:44:44.102424 18158 tablet_bootstrap.cc:492] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: Bootstrap starting.
I20260925 00:44:44.105046 18158 tablet_bootstrap.cc:654] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: Neither blocks nor log segments found. Creating new log.
I20260925 00:44:44.106015 18158 log.cc:824] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: Log is configured to *not* fsync() on all Append() calls
I20260925 00:44:44.108357 18158 tablet_bootstrap.cc:492] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: No bootstrap required, opened a new log
I20260925 00:44:44.108574 18158 ts_tablet_manager.cc:1397] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:44:44.111279 18154 tablet_bootstrap.cc:492] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d: No bootstrap required, opened a new log
I20260925 00:44:44.111479 18154 ts_tablet_manager.cc:1397] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260925 00:44:44.113345 18158 raft_consensus.cc:348] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43813 } } peers { permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 } } peers { permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 } }
I20260925 00:44:44.113613 18158 raft_consensus.cc:374] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:44:44.113659 18158 raft_consensus.cc:729] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3b8d94c60b465bb9b0ca0882e70810, State: Initialized, Role: FOLLOWER
I20260925 00:44:44.114411 18158 consensus_queue.cc:260] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43813 } } peers { permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 } } peers { permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 } }
I20260925 00:44:44.116077 18158 ts_tablet_manager.cc:1428] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260925 00:44:44.116545 18001 heartbeater.cc:503] Master 127.6.72.126:46173 was elected leader, sending a full tablet report...
I20260925 00:44:44.118682 18155 tablet_bootstrap.cc:492] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: No bootstrap required, opened a new log
I20260925 00:44:44.118954 18155 ts_tablet_manager.cc:1397] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.000s
I20260925 00:44:44.121027 18154 raft_consensus.cc:348] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43813 } } peers { permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 } } peers { permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 } }
I20260925 00:44:44.121322 18154 raft_consensus.cc:374] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:44:44.121381 18154 raft_consensus.cc:729] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc50dd4f524498a0cd1c42af878b7d, State: Initialized, Role: FOLLOWER
I20260925 00:44:44.122958 18154 consensus_queue.cc:260] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43813 } } peers { permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 } } peers { permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 } }
I20260925 00:44:44.123977 18154 ts_tablet_manager.cc:1428] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260925 00:44:44.126480 17864 heartbeater.cc:503] Master 127.6.72.126:46173 was elected leader, sending a full tablet report...
W20260925 00:44:44.126869 17865 tablet.cc:2369] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:44:44.129638 18155 raft_consensus.cc:348] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43813 } } peers { permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 } } peers { permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 } }
I20260925 00:44:44.134196 18155 raft_consensus.cc:374] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:44:44.134335 18155 raft_consensus.cc:729] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb0f5ebe6e3f4a139d86f3a5b4daf61a, State: Initialized, Role: FOLLOWER
I20260925 00:44:44.135000 18155 consensus_queue.cc:260] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43813 } } peers { permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 } } peers { permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 } }
I20260925 00:44:44.136271 18155 ts_tablet_manager.cc:1428] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260925 00:44:44.136649 18135 heartbeater.cc:503] Master 127.6.72.126:46173 was elected leader, sending a full tablet report...
Sep 25 00:44:44 dist-test-slave-w8nh krb5kdc[17175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297083, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
W20260925 00:44:44.179128 18136 tablet.cc:2369] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:44:44.272027 18002 tablet.cc:2369] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:44:44.291069 18160 raft_consensus.cc:482] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:44:44.291332 18160 raft_consensus.cc:504] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43813 } } peers { permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 } } peers { permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 } }
I20260925 00:44:44.292796 18160 leader_election.cc:288] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb0f5ebe6e3f4a139d86f3a5b4daf61a (127.6.72.67:32837), d5fc50dd4f524498a0cd1c42af878b7d (127.6.72.65:35397)
I20260925 00:44:44.305364 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.294256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:42903 (local address 127.6.72.67:32837)
0925 00:44:44.294431 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:44.294437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:44.295657 (+ 1220us) server_negotiation.cc:407] Connection header received
0925 00:44:44.296337 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:44.296341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:44.296402 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:44.296459 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:44:44.296910 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.297700 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.301455 (+ 3755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.302090 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.302171 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.302241 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:44.305117 (+ 2876us) server_negotiation.cc:299] Negotiation successful
0925 00:44:44.305209 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 00:44:44.306067 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.293280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:32837 (local address 127.6.72.66:42903)
0925 00:44:44.295560 (+ 2280us) negotiation.cc:107] Waiting for socket to connect
0925 00:44:44.295576 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:44:44.295744 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:44:44.296612 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:44:44.296615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:44:44.296633 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:44:44.296786 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:44.296793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.299452 (+ 2659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.299456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:44.300896 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:44.300908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.304805 (+ 3897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.304808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:44.304965 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.304993 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 00:44:44.305819 (+ 826us) client_negotiation.cc:240] Negotiation successful
0925 00:44:44.305903 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2205,"thread_start_us":131,"threads_started":1}
I20260925 00:44:44.306900 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.294404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:52219 (local address 127.6.72.65:35397)
0925 00:44:44.294550 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:44.294555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:44.296727 (+ 2172us) server_negotiation.cc:407] Connection header received
0925 00:44:44.302811 (+ 6084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:44.302825 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:44.302887 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:44.302941 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:44:44.303356 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.304155 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.305610 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.306206 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.306282 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.306355 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:44.306687 (+ 332us) server_negotiation.cc:299] Negotiation successful
0925 00:44:44.306768 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 00:44:44.306991 18089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9663738cead849d2906d679ab1568e53" candidate_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" is_pre_election: true
I20260925 00:44:44.307376 18089 raft_consensus.cc:2454] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d3b8d94c60b465bb9b0ca0882e70810 in term 0.
I20260925 00:44:44.307440 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.293721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35397 (local address 127.6.72.66:52219)
0925 00:44:44.296641 (+ 2920us) negotiation.cc:107] Waiting for socket to connect
0925 00:44:44.296654 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:44:44.302650 (+ 5996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:44:44.303086 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:44:44.303089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:44:44.303107 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:44:44.303228 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:44.303233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.304364 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.304368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:44.305468 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:44:44.305478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.306452 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.306455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:44:44.306548 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.306570 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:44:44.307225 (+ 655us) client_negotiation.cc:240] Negotiation successful
0925 00:44:44.307302 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2858,"thread_start_us":88,"threads_started":1}
I20260925 00:44:44.307928 17890 leader_election.cc:302] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3b8d94c60b465bb9b0ca0882e70810, bb0f5ebe6e3f4a139d86f3a5b4daf61a; no voters:
I20260925 00:44:44.308036 17800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9663738cead849d2906d679ab1568e53" candidate_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" is_pre_election: true
I20260925 00:44:44.308311 17800 raft_consensus.cc:2454] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d3b8d94c60b465bb9b0ca0882e70810 in term 0.
I20260925 00:44:44.308283 18160 raft_consensus.cc:2781] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:44:44.308380 18160 raft_consensus.cc:482] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:44:44.308425 18160 raft_consensus.cc:3037] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:44:44.309845 18160 raft_consensus.cc:504] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43813 } } peers { permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 } } peers { permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 } }
I20260925 00:44:44.310231 18160 leader_election.cc:288] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [CANDIDATE]: Term 1 election: Requested vote from peers bb0f5ebe6e3f4a139d86f3a5b4daf61a (127.6.72.67:32837), d5fc50dd4f524498a0cd1c42af878b7d (127.6.72.65:35397)
I20260925 00:44:44.310642 17800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9663738cead849d2906d679ab1568e53" candidate_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc50dd4f524498a0cd1c42af878b7d"
I20260925 00:44:44.310753 17800 raft_consensus.cc:3037] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:44:44.311968 17800 raft_consensus.cc:2454] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d3b8d94c60b465bb9b0ca0882e70810 in term 1.
I20260925 00:44:44.312623 17888 leader_election.cc:302] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3b8d94c60b465bb9b0ca0882e70810, d5fc50dd4f524498a0cd1c42af878b7d; no voters:
I20260925 00:44:44.313174 18089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9663738cead849d2906d679ab1568e53" candidate_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a"
I20260925 00:44:44.313304 18089 raft_consensus.cc:3037] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:44:44.314585 18089 raft_consensus.cc:2454] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d3b8d94c60b465bb9b0ca0882e70810 in term 1.
I20260925 00:44:44.315168 18160 raft_consensus.cc:2781] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:44:44.315424 18160 raft_consensus.cc:686] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 1 LEADER]: Becoming Leader. State: Replica: 9d3b8d94c60b465bb9b0ca0882e70810, State: Running, Role: LEADER
I20260925 00:44:44.315784 18160 consensus_queue.cc:237] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43813 } } peers { permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 } } peers { permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 } }
I20260925 00:44:44.320401 17650 catalog_manager.cc:5495] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d3b8d94c60b465bb9b0ca0882e70810 (127.6.72.66). New cstate: current_term: 1 leader_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d3b8d94c60b465bb9b0ca0882e70810" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 43813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 } health_report { overall_health: UNKNOWN } } }
I20260925 00:44:44.400599 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.395504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.6.72.66:43813)
0925 00:44:44.395707 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 00:44:44.395712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:44:44.395726 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:44:44.395822 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:44:44.395825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:44:44.395890 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:44:44.395998 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:44:44.396396 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.397169 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.398410 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:44:44.398763 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:44:44.398857 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:44:44.399266 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:44:44.399836 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:44:44.399951 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 00:44:44.400381 (+ 430us) server_negotiation.cc:299] Negotiation successful
0925 00:44:44.400438 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260925 00:44:44.412833 17800 raft_consensus.cc:1261] T 9663738cead849d2906d679ab1568e53 P d5fc50dd4f524498a0cd1c42af878b7d [term 1 FOLLOWER]: Refusing update from remote peer 9d3b8d94c60b465bb9b0ca0882e70810: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:44:44.413688 18167 consensus_queue.cc:1035] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc50dd4f524498a0cd1c42af878b7d" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:44:44.434063 18171 mvcc.cc:205] Tried to move back new op lower bound from 7333056857745629184 to 7333056857361108992. Current Snapshot: MvccSnapshot[applied={T|T < 7333056857745629184}]
I20260925 00:44:44.435464 18089 raft_consensus.cc:1261] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a [term 1 FOLLOWER]: Refusing update from remote peer 9d3b8d94c60b465bb9b0ca0882e70810: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:44:44.436192 18160 consensus_queue.cc:1035] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb0f5ebe6e3f4a139d86f3a5b4daf61a" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 32837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:44:44.446764 18178 mvcc.cc:205] Tried to move back new op lower bound from 7333056857745629184 to 7333056857361108992. Current Snapshot: MvccSnapshot[applied={T|T < 7333056857745629184}]
I20260925 00:44:44.448880 18170 mvcc.cc:205] Tried to move back new op lower bound from 7333056857745629184 to 7333056857361108992. Current Snapshot: MvccSnapshot[applied={T|T < 7333056857745629184 or (T in {7333056857745629184})}]
I20260925 00:44:44.448937 17650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:44:44.449112 17650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:44:44.454324 17650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07d6f7fd0ca948f7b26db231a19ebe04: Sending DeleteTablet for 3 replicas of tablet 9663738cead849d2906d679ab1568e53
I20260925 00:44:44.455379 17932 tablet_service.cc:1514] Processing DeleteTablet for tablet 9663738cead849d2906d679ab1568e53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:44:44 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:47338
I20260925 00:44:44.456321 18069 tablet_service.cc:1514] Processing DeleteTablet for tablet 9663738cead849d2906d679ab1568e53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:44:44 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:49982
I20260925 00:44:44.457363 17791 tablet_service.cc:1514] Processing DeleteTablet for tablet 9663738cead849d2906d679ab1568e53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:44:44 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:41194
I20260925 00:44:44.457883 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 17732
I20260925 00:44:44.461644 18189 tablet_replica.cc:330] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: stopping tablet replica
I20260925 00:44:44.461935 18188 tablet_replica.cc:330] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: stopping tablet replica
I20260925 00:44:44.467675 18188 raft_consensus.cc:2229] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:44:44.468122 18189 raft_consensus.cc:2229] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:44:44.468566 18189 raft_consensus.cc:2258] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:44:44.469074 18188 raft_consensus.cc:2258] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:44:44.471062 18189 ts_tablet_manager.cc:1905] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:44:44.475585 18189 ts_tablet_manager.cc:1918] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:44:44.472667 18188 ts_tablet_manager.cc:1905] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:44:44.478376 18188 ts_tablet_manager.cc:1918] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:44:44.478565 18188 log.cc:1196] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/9663738cead849d2906d679ab1568e53
W20260925 00:44:44.479355 17635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:35397 (error 108)
I20260925 00:44:44.480046 18189 log.cc:1196] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/9663738cead849d2906d679ab1568e53
I20260925 00:44:44.480799 18189 ts_tablet_manager.cc:1939] T 9663738cead849d2906d679ab1568e53 P bb0f5ebe6e3f4a139d86f3a5b4daf61a: Deleting consensus metadata
I20260925 00:44:44.481067 18188 ts_tablet_manager.cc:1939] T 9663738cead849d2906d679ab1568e53 P 9d3b8d94c60b465bb9b0ca0882e70810: Deleting consensus metadata
I20260925 00:44:44.481428 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 17871
I20260925 00:44:44.484853 17637 catalog_manager.cc:4841] TS bb0f5ebe6e3f4a139d86f3a5b4daf61a (127.6.72.67:32837): tablet 9663738cead849d2906d679ab1568e53 (table test-table [id=be87ab9a1e9b4505bbafcf3cf1d8d56e]) successfully deleted
I20260925 00:44:44.486027 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.485671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35397 (local address 127.0.0.1:41196)
0925 00:44:44.485869 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 00:44:44.485942 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:35397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260925 00:44:44.486369 17635 catalog_manager.cc:4584] TS d5fc50dd4f524498a0cd1c42af878b7d (127.6.72.65:35397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9663738cead849d2906d679ab1568e53: Network error: Client connection negotiation failed: client connection to 127.6.72.65:35397: connect: Connection refused (error 111)
I20260925 00:44:44.488662 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 18006
I20260925 00:44:44.490509 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:44:44.488866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:43813 (local address 127.0.0.1:47366)
0925 00:44:44.490361 (+ 1495us) negotiation.cc:107] Waiting for socket to connect
0925 00:44:44.490416 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.66:43813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1415}
I20260925 00:44:44.495729 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 17619
I20260925 00:44:44.503880 6433 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 00:44:44.849097 6433 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 00:44:44.849229 6433 mini_ranger.cc:66] Stopping Ranger...
I20260925 00:44:45.195473 6433 mini_ranger.cc:68] Stopped Ranger
2026-09-25 00:44:45.195 UTC [17205] LOG: received smart shutdown request
2026-09-25 00:44:45.199 UTC [17205] LOG: background worker "logical replication launcher" (PID 17214) exited with exit code 1
2026-09-25 00:44:45.200 UTC [17209] LOG: shutting down
2026-09-25 00:44:45.391 UTC [17205] LOG: database system is shut down
2026-09-25T00:44:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48459 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:44:45 dist-test-slave-w8nh krb5kdc[18200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:44:45 dist-test-slave-w8nh krb5kdc[18200](info): set up 2 sockets
Sep 25 00:44:45 dist-test-slave-w8nh krb5kdc[18200](info): commencing operation
krb5kdc: starting...
W20260925 00:44:46.552410 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:44:46 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:44:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:44:46Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.72.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.72.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/rangeradmin_127.6.72.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/rangeradmin_127.6.72.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.72.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.72.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/rangerlookup_127.6.72.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/rangerlookup_127.6.72.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.72.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.72.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/HTTP_127.6.72.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/HTTP_127.6.72.84@KRBTEST.COM.keytab.
I20260925 00:44:46.730782 6433 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasklhk22w/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/postgres -l logfile start
2026-09-25 00:44:49.414 UTC [18230] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 00:44:49.414 UTC [18230] LOG: listening on IPv4 address "127.6.72.84", port 55243
2026-09-25 00:44:49.418 UTC [18230] LOG: listening on Unix socket "/tmp/.s.PGSQL.55243"
2026-09-25 00:44:49.439 UTC [18233] LOG: database system was shut down at 2026-09-25 00:44:47 UTC
2026-09-25 00:44:49.443 UTC [18230] LOG: database system is ready to accept connections
I20260925 00:44:50.400211 6433 mini_postgres.cc:96] Postgres bound to 55243
2026-09-25 00:44:50.406 UTC [18241] FATAL: database "slave" does not exist
127.6.72.84:55243 - accepting connections
I20260925 00:44:50.407502 6433 mini_ranger.cc:161] Starting Ranger...
I20260925 00:44:50.422848 6433 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 00:44:50.945091 6433 mini_ranger.cc:87] Created ranger Postgres database
I20260925 00:44:50.945187 6433 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-admin
2026-09-25 00:44:51,249 [I] DB FLAVOR :POSTGRES
2026-09-25 00:44:51,250 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 00:44:51,250 [I] Checking connection..
2026-09-25 00:44:51,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 00:44:51,518 [I] Checking connection passed.
2026-09-25 00:44:51,518 [I] --------- Verifying version history table ---------
2026-09-25 00:44:51,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 00:44:51,759 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 00:44:51,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 00:44:52,019 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 00:44:52,019 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 00:44:52,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 00:44:52.264 UTC [18305] WARNING: there is no transaction in progress
2026-09-25 00:44:52,270 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 00:44:52,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 00:44:52,535 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 00:44:52,535 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 00:44:52,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 00:44:52,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 00:44:53,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-w8nh.c.gcp-upstream.internal','N') ;"
2026-09-25 00:44:53,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 00:44:53,571 [I] Table x_portal_user does not exist in database ranger
2026-09-25 00:44:53,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 00:44:53,807 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 00:44:53,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 00:44:54,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 00:44:54,301 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 00:44:54,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 00:44:55.355 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.373 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.390 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.408 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.424 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.443 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.503 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.512 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.529 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.540 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.551 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.566 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.576 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.583 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.590 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:55.597 UTC [18413] WARNING: there is no transaction in progress
2026-09-25 00:44:56,329 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 00:44:56,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 00:44:56,578 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 00:44:56,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 00:44:56,816 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 00:44:56,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 00:44:57,059 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 00:44:57,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 00:44:57,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 00:44:57,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-w8nh.c.gcp-upstream.internal';"
2026-09-25 00:44:57,861 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 00:44:57,861 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 00:44:58,107 [I] DB_PATCHES have already been applied
I20260925 00:44:58.114073 6433 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-admin:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasklhk22w/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 00:44:58.114146 6433 mini_ranger.cc:193] Using host: 127.6.72.84
I20260925 00:44:58.117673 6433 mini_ranger.cc:227] Ranger admin URL: http://127.6.72.84:56529
Sep 25, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 12:44:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 12:44:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 00:44:59.290815 6433 mini_ranger.cc:160] Time spent starting Ranger: real 8.883s user 0.003s sys 0.007s
Sep 25, 2026 12:45:19 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 12:45:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Sep 25, 2026 12:45:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 12:45:19 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 12:45:19 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 00:45:20.577950 6433 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.72.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.72.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/rangerkms_127.6.72.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.72.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/rangerkms_127.6.72.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.72.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.72.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.72.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/HTTP_127.6.72.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.72.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/HTTP_127.6.72.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 00:45:20 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297120, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 00:45:20.682003 6433 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 00:45:20.692150 6433 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 00:45:21.665795 6433 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 00:45:21.665868 6433 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-kms
I20260925 00:45:21.665882 6433 mini_ranger_kms.cc:227] Using postgres at 127.6.72.84:55243
2026-09-25 00:45:21,839 [I] DB FLAVOR :POSTGRES
2026-09-25 00:45:21,839 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 00:45:21,840 [I] Checking connection
2026-09-25 00:45:21,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 00:45:22,071 [I] connection success
2026-09-25 00:45:22,071 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 00:45:22,071 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 00:45:22,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 00:45:22,310 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 00:45:22,310 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 00:45:22,310 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 00:45:22,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.72.84:55243/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 00:45:22,573 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 00:45:23.015862 6433 mini_ranger_kms.cc:310] Created kms service
I20260925 00:45:23.223727 6433 mini_ranger_kms.cc:326] Created kudu user
I20260925 00:45:23.268455 6433 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 00:45:23 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297120, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:23.665747 6433 mini_ranger_kms.cc:384] Added ranger policy
I20260925 00:45:23.665987 6433 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-kms:/tmp/dist-test-tasklhk22w/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasklhk22w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 00:45:23.666013 6433 mini_ranger_kms.cc:240] Using host: 127.6.72.84
I20260925 00:45:23.670367 6433 mini_ranger_kms.cc:276] Ranger KMS PID: 18593
I20260925 00:45:23.670465 6433 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.72.84:50655
Sep 25, 2026 12:45:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 12:45:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 12:45:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 12:45:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 12:45:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 12:45:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 00:45:25.943436 6433 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.261s user 0.023s sys 0.134s
I20260925 00:45:25.943588 6433 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 00:45:25.943676 6433 mini_ranger_kms.cc:401] 127.6.72.84:50655/kms/v1/keys
Sep 25 00:45:26 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790297126, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.72.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 00:45:26 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297126, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.72.84@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
Sep 25 00:45:26 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297126, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.72.84@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
Sep 25 00:45:29 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:45:29.152486 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:36701
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:36701
--ranger_config_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:50655/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:29.271045 18644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:29.271358 18644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:29.271400 18644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:29.275665 18644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:45:29.275739 18644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:29.275760 18644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:29.275776 18644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:45:29.275791 18644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:45:29.281838 18644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.72.84:50655/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:36701
--ranger_config_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:36701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:29.283015 18644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:29.284049 18644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:29.292891 18649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:29.292991 18652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:29.293120 18644 server_base.cc:1034] running on GCE node
W20260925 00:45:29.292896 18650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:29.293844 18644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:29.295012 18644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:29.296197 18644 hybrid_clock.cc:648] HybridClock initialized: now 1790297129296170 us; error 45 us; skew 500 ppm
Sep 25 00:45:29 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297129, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:29.304770 18644 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:45:29.305819 18644 webserver.cc:460] Webserver started at http://127.6.72.126:35989/ using document root <none> and password file <none>
I20260925 00:45:29.306382 18644 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:29.306439 18644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:29.306622 18644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 00:45:29 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297129, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:29.353541 18644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "d415838ca7444881b95c2d6337518420"
format_stamp: "Formatted at 2026-09-25 00:45:29 on dist-test-slave-w8nh"
server_key: "a1d5b7f07c5d0c33169c46915aeab297"
server_key_iv: "d9f1997c9cad44216310000b1b4fb700"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:29.354162 18644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "d415838ca7444881b95c2d6337518420"
format_stamp: "Formatted at 2026-09-25 00:45:29 on dist-test-slave-w8nh"
server_key: "a1d5b7f07c5d0c33169c46915aeab297"
server_key_iv: "d9f1997c9cad44216310000b1b4fb700"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:29.358305 18644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:45:29.376788 18659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:29.378371 18644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:45:29.378530 18644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "d415838ca7444881b95c2d6337518420"
format_stamp: "Formatted at 2026-09-25 00:45:29 on dist-test-slave-w8nh"
server_key: "a1d5b7f07c5d0c33169c46915aeab297"
server_key_iv: "d9f1997c9cad44216310000b1b4fb700"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:29.378676 18644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:29.394304 18644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:29.397442 18644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:29.397676 18644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:29.404870 18644 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:36701
I20260925 00:45:29.404879 18711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:36701 every 8 connection(s)
I20260925 00:45:29.405838 18644 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:45:29.408659 18712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:29.410436 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 18644
I20260925 00:45:29.410597 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:45:29.410905 6433 external_mini_cluster.cc:1428] Setting key 8bff9dda567726193cb66cbb70c098bd
I20260925 00:45:29.414552 18712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: Bootstrap starting.
I20260925 00:45:29.417023 18712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:29.417811 18712 log.cc:824] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:29.419736 18712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: No bootstrap required, opened a new log
Sep 25 00:45:29 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:29.423247 18712 raft_consensus.cc:348] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } }
I20260925 00:45:29.423539 18712 raft_consensus.cc:374] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:29.423614 18712 raft_consensus.cc:729] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d415838ca7444881b95c2d6337518420, State: Initialized, Role: FOLLOWER
I20260925 00:45:29.424136 18712 consensus_queue.cc:260] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } }
I20260925 00:45:29.424342 18712 raft_consensus.cc:388] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:45:29.424407 18712 raft_consensus.cc:482] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:45:29.424501 18712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:29.425614 18712 raft_consensus.cc:504] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } }
I20260925 00:45:29.425951 18712 leader_election.cc:302] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d415838ca7444881b95c2d6337518420; no voters:
I20260925 00:45:29.426266 18712 leader_election.cc:288] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:45:29.426306 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:29.412426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40092 (local address 127.6.72.126:36701)
0925 00:45:29.412810 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:29.412834 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:29.412868 (+ 34us) server_negotiation.cc:407] Connection header received
0925 00:45:29.413680 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:29.413698 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:29.414021 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:29.414297 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:29.415201 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:29.416215 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:29.417456 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:29.418029 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:29.418187 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:29.421438 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:29.421445 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:29.421458 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:29.421483 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:29.423110 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:29.423649 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:29.423655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:29.423660 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:29.423823 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:29.424133 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:29.424135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:29.424136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:29.424749 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:29.424945 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:29.425169 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 00:45:29.425435 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":140,"threads_started":1}
I20260925 00:45:29.426371 18717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:45:29.426599 18717 raft_consensus.cc:686] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 1 LEADER]: Becoming Leader. State: Replica: d415838ca7444881b95c2d6337518420, State: Running, Role: LEADER
I20260925 00:45:29.426893 18717 consensus_queue.cc:237] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } }
I20260925 00:45:29.427479 18712 sys_catalog.cc:564] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:45:29.428027 18717 sys_catalog.cc:455] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d415838ca7444881b95c2d6337518420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } } }
I20260925 00:45:29.428220 18717 sys_catalog.cc:458] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:29.428480 18717 sys_catalog.cc:455] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d415838ca7444881b95c2d6337518420. Latest consensus state: current_term: 1 leader_uuid: "d415838ca7444881b95c2d6337518420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } } }
I20260925 00:45:29.428838 18717 sys_catalog.cc:458] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:29.429203 18725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:45:29.431898 18725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:45:29.432354 18712 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 00:45:29.441870 18725 catalog_manager.cc:1347] Generated new cluster ID: b2fa59ac0c4f4249b824ea1bfb6de6c3
I20260925 00:45:29.441954 18725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:45:29.457006 18725 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:45:29.458463 18725 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:45:29.469147 18725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: Generated new TSK 0
I20260925 00:45:29.470114 18725 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:45:30.164544 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:50655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36701
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:30.286953 18757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:30.287259 18757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:30.287338 18757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:30.293318 18757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:30.293418 18757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:30.293540 18757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:45:30.300410 18757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.72.84:50655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:30.302006 18757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:30.303177 18757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:30.313926 18762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:30.313925 18763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:30.314464 18765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:30.315004 18757 server_base.cc:1034] running on GCE node
I20260925 00:45:30.315490 18757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:30.316423 18757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:30.317627 18757 hybrid_clock.cc:648] HybridClock initialized: now 1790297130317590 us; error 55 us; skew 500 ppm
Sep 25 00:45:30 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297130, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:30.320765 18757 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:45:30.321903 18757 webserver.cc:460] Webserver started at http://127.6.72.65:35587/ using document root <none> and password file <none>
I20260925 00:45:30.322523 18757 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:30.322619 18757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:30.322870 18757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 00:45:30 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297130, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:30.342264 18757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "95b572ca25cb41309320414d1219d9b1"
format_stamp: "Formatted at 2026-09-25 00:45:30 on dist-test-slave-w8nh"
server_key: "22dc68582065d7ecb17c64a17a7fceda"
server_key_iv: "b6554610263c6cc30c315dd001185d15"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:30.343129 18757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "95b572ca25cb41309320414d1219d9b1"
format_stamp: "Formatted at 2026-09-25 00:45:30 on dist-test-slave-w8nh"
server_key: "22dc68582065d7ecb17c64a17a7fceda"
server_key_iv: "b6554610263c6cc30c315dd001185d15"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:30.347726 18757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:45:30.371279 18774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:30.373154 18757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:45:30.373304 18757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "95b572ca25cb41309320414d1219d9b1"
format_stamp: "Formatted at 2026-09-25 00:45:30 on dist-test-slave-w8nh"
server_key: "22dc68582065d7ecb17c64a17a7fceda"
server_key_iv: "b6554610263c6cc30c315dd001185d15"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:30.373467 18757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:30.392592 18757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:30.396453 18757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:30.396674 18757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:30.397454 18757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:30.398619 18757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:30.398686 18757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:30.398744 18757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:30.398777 18757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:30.410050 18757 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:40223
I20260925 00:45:30.411335 18757 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:45:30.412912 18887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:40223 every 8 connection(s)
I20260925 00:45:30.415642 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 18757
I20260925 00:45:30.415752 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:45:30.416109 6433 external_mini_cluster.cc:1428] Setting key 08f642720a4ffdc69b564e8b5055e4f0
Sep 25 00:45:30 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297130, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:45:30.442164 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:30.421724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:46231 (local address 127.6.72.126:36701)
0925 00:45:30.422011 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:30.422020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:30.426108 (+ 4088us) server_negotiation.cc:407] Connection header received
0925 00:45:30.426163 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:30.426166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:30.426224 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:30.426309 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:30.427871 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:30.428919 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:30.430932 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:30.431399 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:30.431499 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:30.434829 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:30.434835 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:30.434838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:30.434870 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:30.436868 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:30.437720 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:30.437723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:30.437725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:30.437820 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:30.438247 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:30.438250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:30.438252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:30.438489 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:30.438692 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:30.441802 (+ 3110us) server_negotiation.cc:299] Negotiation successful
0925 00:45:30.441956 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":115,"threads_started":1}
I20260925 00:45:30.444262 18891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:30.420675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36701 (local address 127.6.72.65:46231)
0925 00:45:30.424610 (+ 3935us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:30.424680 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:30.425788 (+ 1108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:30.426640 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:30.426650 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:30.427015 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:30.427714 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:30.427734 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:30.429087 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:30.429093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:30.430739 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:30.430756 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:30.431760 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:30.431765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:30.431994 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:30.432773 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:30.432798 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:30.434590 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:30.437029 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:30.437036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:30.437069 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:30.437434 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:30.437936 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:30.437939 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:30.437941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:30.438135 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:30.438787 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:30.438794 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:30.439174 (+ 380us) client_negotiation.cc:769] Sending connection context
0925 00:45:30.442682 (+ 3508us) client_negotiation.cc:240] Negotiation successful
0925 00:45:30.443089 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3670,"thread_start_us":160,"threads_started":1}
I20260925 00:45:30.446132 18888 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36701
I20260925 00:45:30.446396 18888 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:30.447027 18888 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
I20260925 00:45:30.448917 18676 ts_manager.cc:194] Registered new tserver with Master: 95b572ca25cb41309320414d1219d9b1 (127.6.72.65:40223)
I20260925 00:45:30.450537 18676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:46231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:45:30.467573 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:50655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36701
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 00:45:30 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790297130, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 00:45:30.625478 18896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:30.625878 18896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:30.625944 18896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:30.632225 18896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:30.632350 18896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:30.632443 18896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:45:30.640681 18896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.72.84:50655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:30.642073 18896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:30.643165 18896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:30.654654 18905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:30.654739 18902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:30.655172 18903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:30.655550 18896 server_base.cc:1034] running on GCE node
I20260925 00:45:30.656095 18896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:30.656991 18896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:30.658200 18896 hybrid_clock.cc:648] HybridClock initialized: now 1790297130658171 us; error 41 us; skew 500 ppm
Sep 25 00:45:30 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297130, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:30.662051 18896 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:45:30.663197 18896 webserver.cc:460] Webserver started at http://127.6.72.66:34325/ using document root <none> and password file <none>
I20260925 00:45:30.663928 18896 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:30.664013 18896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:30.664278 18896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 00:45:30 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297130, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:30.681754 18896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "98af5b6c261440dfae9c306ec84822c2"
format_stamp: "Formatted at 2026-09-25 00:45:30 on dist-test-slave-w8nh"
server_key: "b3d137ae477a5f8b397a87ed1dae3bff"
server_key_iv: "d973e7ba2ebe5cf69b0c23b6a3d49b6e"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:30.682683 18896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "98af5b6c261440dfae9c306ec84822c2"
format_stamp: "Formatted at 2026-09-25 00:45:30 on dist-test-slave-w8nh"
server_key: "b3d137ae477a5f8b397a87ed1dae3bff"
server_key_iv: "d973e7ba2ebe5cf69b0c23b6a3d49b6e"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:30.688102 18896 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 00:45:30.704345 18912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:30.706115 18896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:45:30.706269 18896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "98af5b6c261440dfae9c306ec84822c2"
format_stamp: "Formatted at 2026-09-25 00:45:30 on dist-test-slave-w8nh"
server_key: "b3d137ae477a5f8b397a87ed1dae3bff"
server_key_iv: "d973e7ba2ebe5cf69b0c23b6a3d49b6e"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:30.706413 18896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:30.748301 18896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:30.752067 18896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:30.752321 18896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:30.753095 18896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:30.754407 18896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:30.754482 18896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:30.754552 18896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:30.754588 18896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:30.767710 18896 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:45313
I20260925 00:45:30.768959 18896 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:45:30.769035 19025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:45313 every 8 connection(s)
I20260925 00:45:30.769341 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 18896
I20260925 00:45:30.769485 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:45:30.769842 6433 external_mini_cluster.cc:1428] Setting key 99fb1d846d5075a11350adc7378411d5
Sep 25 00:45:30 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297130, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:45:30.797444 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:30.775427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:48597 (local address 127.6.72.126:36701)
0925 00:45:30.775578 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:30.775583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:30.776852 (+ 1269us) server_negotiation.cc:407] Connection header received
0925 00:45:30.778829 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:30.778833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:30.778893 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:30.778963 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:30.780823 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:30.781706 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:30.782991 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:30.783368 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:30.783496 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:30.788899 (+ 5403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:30.788904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:30.788907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:30.788943 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:30.790854 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:30.793718 (+ 2864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:30.793722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:30.793724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:30.793820 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:30.794282 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:30.794285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:30.794287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:30.794546 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:30.794637 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:30.797116 (+ 2479us) server_negotiation.cc:299] Negotiation successful
0925 00:45:30.797273 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 00:45:30.799255 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:30.774207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36701 (local address 127.6.72.66:48597)
0925 00:45:30.776612 (+ 2405us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:30.776678 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:30.777917 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:30.779404 (+ 1487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:30.779414 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:30.779870 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:30.780641 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:30.780660 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:30.781858 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:30.781861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:30.782873 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:30.782882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:30.783563 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:30.783568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:30.783845 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:30.784650 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:30.784677 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:30.788631 (+ 3954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:30.791020 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:30.791028 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:30.791050 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:30.793576 (+ 2526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:30.793944 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:30.793947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:30.793950 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:30.794159 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:30.794709 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:30.794716 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:30.795087 (+ 371us) client_negotiation.cc:769] Sending connection context
0925 00:45:30.797723 (+ 2636us) client_negotiation.cc:240] Negotiation successful
0925 00:45:30.798104 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2145,"thread_start_us":169,"threads_started":1}
I20260925 00:45:30.800702 19026 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36701
I20260925 00:45:30.800954 19026 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:30.801564 19026 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
I20260925 00:45:30.802701 18676 ts_manager.cc:194] Registered new tserver with Master: 98af5b6c261440dfae9c306ec84822c2 (127.6.72.66:45313)
I20260925 00:45:30.803539 18676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:48597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:45:30.815878 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:50655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36701
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:30.979990 19031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:30.980798 19031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:30.980876 19031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Sep 25 00:45:30 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297130, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
W20260925 00:45:30.986908 19031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:30.986991 19031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:30.987075 19031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:45:30.994917 19031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.72.84:50655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:30.996338 19031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:30.997682 19031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:31.012096 19037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:31.012512 19036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:31.012825 19031 server_base.cc:1034] running on GCE node
W20260925 00:45:31.012914 19039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:31.013489 19031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:31.014359 19031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:31.015743 19031 hybrid_clock.cc:648] HybridClock initialized: now 1790297131015673 us; error 84 us; skew 500 ppm
Sep 25 00:45:31 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:31.019511 19031 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:45:31.020767 19031 webserver.cc:460] Webserver started at http://127.6.72.67:46865/ using document root <none> and password file <none>
I20260925 00:45:31.021517 19031 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:31.021612 19031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:31.021857 19031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 00:45:31 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:31.042277 19031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "2cb93cdcadeb451896dee2098789cfdc"
format_stamp: "Formatted at 2026-09-25 00:45:31 on dist-test-slave-w8nh"
server_key: "bcbdc2b839ba603ceffd763d53529610"
server_key_iv: "85062fb343e02caa9dbbcd905e47f22c"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:31.047370 19031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "2cb93cdcadeb451896dee2098789cfdc"
format_stamp: "Formatted at 2026-09-25 00:45:31 on dist-test-slave-w8nh"
server_key: "bcbdc2b839ba603ceffd763d53529610"
server_key_iv: "85062fb343e02caa9dbbcd905e47f22c"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:31.053287 19031 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260925 00:45:31.089303 19046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:31.091162 19031 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260925 00:45:31.091320 19031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "2cb93cdcadeb451896dee2098789cfdc"
format_stamp: "Formatted at 2026-09-25 00:45:31 on dist-test-slave-w8nh"
server_key: "bcbdc2b839ba603ceffd763d53529610"
server_key_iv: "85062fb343e02caa9dbbcd905e47f22c"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:31.091499 19031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:31.124006 19031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:31.128984 19031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:31.129282 19031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:31.130152 19031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:31.131568 19031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:31.131639 19031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:31.131706 19031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:31.131772 19031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:31.146529 19031 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:46479
I20260925 00:45:31.147747 19031 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:45:31.148520 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 19031
I20260925 00:45:31.148695 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:45:31.149039 6433 external_mini_cluster.cc:1428] Setting key 9697e89213904a16c5d75c177978bc3a
I20260925 00:45:31.151484 19159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:46479 every 8 connection(s)
Sep 25 00:45:31 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:31.177156 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:31.153995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:55795 (local address 127.6.72.126:36701)
0925 00:45:31.154138 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:31.154143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:31.156397 (+ 2254us) server_negotiation.cc:407] Connection header received
0925 00:45:31.158595 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:31.158599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:31.158659 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:31.158715 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:31.160503 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:31.161334 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:31.162992 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:31.163323 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:31.163400 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:31.170751 (+ 7351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:31.170755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:31.170758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:31.170791 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:31.172713 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:31.175141 (+ 2428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:31.175145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:31.175147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:31.175244 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:31.175693 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:31.175697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:31.175699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:31.175937 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:31.176029 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:31.176829 (+ 800us) server_negotiation.cc:299] Negotiation successful
0925 00:45:31.176970 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 00:45:31.179276 19162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:31.154578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36701 (local address 127.6.72.67:55795)
0925 00:45:31.156211 (+ 1633us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:31.156295 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:31.158320 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:31.159162 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:31.159171 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:31.159569 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:31.160314 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:31.160336 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:31.161507 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:31.161510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:31.162866 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:31.162877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:31.163606 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:31.163609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:31.163798 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:31.168468 (+ 4670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:31.168493 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:31.170509 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:31.172873 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:31.172881 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:31.172901 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:31.175007 (+ 2106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:31.175367 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:31.175370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:31.175372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:31.175570 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:31.176113 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:31.176120 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:31.176523 (+ 403us) client_negotiation.cc:769] Sending connection context
0925 00:45:31.177615 (+ 1092us) client_negotiation.cc:240] Negotiation successful
0925 00:45:31.177982 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1389,"thread_start_us":162,"threads_started":1}
I20260925 00:45:31.181046 19160 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36701
I20260925 00:45:31.181309 19160 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:31.181941 19160 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
I20260925 00:45:31.183249 18676 ts_manager.cc:194] Registered new tserver with Master: 2cb93cdcadeb451896dee2098789cfdc (127.6.72.67:46479)
I20260925 00:45:31.184029 18676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:55795
I20260925 00:45:31.187206 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:45:31.187938 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 18757
I20260925 00:45:31.198748 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 18896
I20260925 00:45:31.207933 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 19031
I20260925 00:45:31.216377 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 18644
I20260925 00:45:31.224704 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:36701
--webserver_interface=127.6.72.126
--webserver_port=35989
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:36701
--ranger_config_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:50655/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.72.84:50655/kms with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:31.360818 19163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:31.361079 19163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:31.361119 19163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:31.365118 19163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:45:31.365173 19163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:31.365192 19163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:31.365209 19163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:45:31.365224 19163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:45:31.370687 19163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.72.84:50655/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:36701
--ranger_config_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:36701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=35989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:31.371706 19163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:31.372877 19163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:31.381217 19169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:31.381282 19163 server_base.cc:1034] running on GCE node
W20260925 00:45:31.381212 19168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:31.381235 19171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:31.381928 19163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:31.383064 19163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:31.384334 19163 hybrid_clock.cc:648] HybridClock initialized: now 1790297131384315 us; error 29 us; skew 500 ppm
Sep 25 00:45:31 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:31.387224 19163 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:45:31.388150 19163 webserver.cc:460] Webserver started at http://127.6.72.126:35989/ using document root <none> and password file <none>
I20260925 00:45:31.388726 19163 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:31.388823 19163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 00:45:31 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:31.448649 19163 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:45:31.450975 19179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:31.452447 19163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:45:31.452581 19163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "d415838ca7444881b95c2d6337518420"
format_stamp: "Formatted at 2026-09-25 00:45:29 on dist-test-slave-w8nh"
server_key: "a1d5b7f07c5d0c33169c46915aeab297"
server_key_iv: "d9f1997c9cad44216310000b1b4fb700"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:31.453153 19163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:31.466260 19163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:31.469561 19163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:31.469780 19163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:31.477131 19163 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:36701
I20260925 00:45:31.477156 19231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:36701 every 8 connection(s)
I20260925 00:45:31.478331 19163 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:45:31.482743 19232 sys_catalog.cc:263] Verifying existing consensus state
I20260925 00:45:31.483989 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 19163
I20260925 00:45:31.484486 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:50655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:40223
--local_ip_for_outbound_sockets=127.6.72.65
--tserver_master_addrs=127.6.72.126:36701
--webserver_port=35587
--webserver_interface=127.6.72.65
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.72.84:50655/kms with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:31.484521 19232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: Bootstrap starting.
I20260925 00:45:31.501145 19232 log.cc:824] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:31.507838 19232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:45:31.508284 19232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: Bootstrap complete.
I20260925 00:45:31.512439 19232 raft_consensus.cc:348] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } }
I20260925 00:45:31.512866 19232 raft_consensus.cc:729] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d415838ca7444881b95c2d6337518420, State: Initialized, Role: FOLLOWER
I20260925 00:45:31.513345 19232 consensus_queue.cc:260] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } }
I20260925 00:45:31.513490 19232 raft_consensus.cc:388] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:45:31.513584 19232 raft_consensus.cc:482] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:45:31.513700 19232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 1 FOLLOWER]: Advancing to term 2
I20260925 00:45:31.515511 19232 raft_consensus.cc:504] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } }
I20260925 00:45:31.515844 19232 leader_election.cc:302] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d415838ca7444881b95c2d6337518420; no voters:
I20260925 00:45:31.516180 19232 leader_election.cc:288] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 00:45:31.516295 19236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 00:45:31.516512 19236 raft_consensus.cc:686] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [term 2 LEADER]: Becoming Leader. State: Replica: d415838ca7444881b95c2d6337518420, State: Running, Role: LEADER
I20260925 00:45:31.516798 19236 consensus_queue.cc:237] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } }
I20260925 00:45:31.517465 19232 sys_catalog.cc:564] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:45:31.517704 19238 sys_catalog.cc:455] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d415838ca7444881b95c2d6337518420. Latest consensus state: current_term: 2 leader_uuid: "d415838ca7444881b95c2d6337518420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } } }
I20260925 00:45:31.517813 19238 sys_catalog.cc:458] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:31.518072 19237 sys_catalog.cc:455] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d415838ca7444881b95c2d6337518420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415838ca7444881b95c2d6337518420" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 36701 } } }
I20260925 00:45:31.518175 19240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:45:31.518182 19237 sys_catalog.cc:458] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420 [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:31.521574 19240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:45:31.522086 19232 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 00:45:31.522284 19240 catalog_manager.cc:1259] Loaded cluster ID: b2fa59ac0c4f4249b824ea1bfb6de6c3
I20260925 00:45:31.522334 19240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:45:31.524101 19240 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:45:31.527027 19240 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: Loaded TSK: 0
I20260925 00:45:31.528613 19240 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 00:45:31.626210 19234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:31.626571 19234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:31.626647 19234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:31.632682 19234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:31.632764 19234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:31.632853 19234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:45:31.640503 19234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.72.84:50655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:40223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=35587
--tserver_master_addrs=127.6.72.126:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:31.641811 19234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:31.643044 19234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 00:45:31.656198 19234 server_base.cc:1034] running on GCE node
W20260925 00:45:31.656399 19272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:31.656999 19270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:31.657223 19269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:31.657718 19234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:31.658499 19234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:31.659693 19234 hybrid_clock.cc:648] HybridClock initialized: now 1790297131659666 us; error 41 us; skew 500 ppm
Sep 25 00:45:31 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:31.663179 19234 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:45:31.664400 19234 webserver.cc:460] Webserver started at http://127.6.72.65:35587/ using document root <none> and password file <none>
I20260925 00:45:31.665226 19234 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:31.665329 19234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 00:45:31 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:31.695168 19234 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.003s sys 0.000s
I20260925 00:45:31.698643 19280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:31.700474 19234 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260925 00:45:31.700635 19234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "95b572ca25cb41309320414d1219d9b1"
format_stamp: "Formatted at 2026-09-25 00:45:30 on dist-test-slave-w8nh"
server_key: "22dc68582065d7ecb17c64a17a7fceda"
server_key_iv: "b6554610263c6cc30c315dd001185d15"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:31.701419 19234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:31.721585 19234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:31.725044 19234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:31.725260 19234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:31.726006 19234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:31.727063 19234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:31.727144 19234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:31.727205 19234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:31.727239 19234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:31.739675 19234 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:40223
I20260925 00:45:31.741026 19234 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:45:31.744529 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 19234
I20260925 00:45:31.745085 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:50655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:45313
--local_ip_for_outbound_sockets=127.6.72.66
--tserver_master_addrs=127.6.72.126:36701
--webserver_port=34325
--webserver_interface=127.6.72.66
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.72.84:50655/kms with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:31.748710 19393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:40223 every 8 connection(s)
Sep 25 00:45:31 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:31.762655 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:31.743672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36701 (local address 127.6.72.65:57891)
0925 00:45:31.744853 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:31.744894 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:31.745811 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:31.747114 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:31.747122 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:31.747469 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:31.748156 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:31.748178 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:31.750125 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:31.750129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:31.751474 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:31.751488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:31.752780 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:31.752784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:31.753027 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:31.754320 (+ 1293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:31.754340 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:31.756044 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:31.758809 (+ 2765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:31.758815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:31.758829 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:31.759196 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:31.759624 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:31.759627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:31.759629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:31.759804 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:31.760676 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:31.760682 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:31.760998 (+ 316us) client_negotiation.cc:769] Sending connection context
0925 00:45:31.761221 (+ 223us) client_negotiation.cc:240] Negotiation successful
0925 00:45:31.761582 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"thread_start_us":144,"threads_started":1}
I20260925 00:45:31.764487 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:31.744912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:57891 (local address 127.6.72.126:36701)
0925 00:45:31.745409 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:31.745430 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:31.745476 (+ 46us) server_negotiation.cc:407] Connection header received
0925 00:45:31.746281 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:31.746299 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:31.746707 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:31.747029 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:31.748368 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:31.749945 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:31.751624 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:31.752395 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:31.752562 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:31.756327 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:31.756335 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:31.756353 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:31.756438 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:31.758667 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:31.759337 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:31.759343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:31.759349 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:31.759500 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:31.759921 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:31.759924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:31.759926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:31.760486 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:31.760618 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:31.763208 (+ 2590us) server_negotiation.cc:299] Negotiation successful
0925 00:45:31.763516 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":131,"threads_started":1}
I20260925 00:45:31.767249 19394 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36701
I20260925 00:45:31.767506 19394 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:31.768133 19394 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
W20260925 00:45:31.770499 19394 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:36701 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 00:45:31.770638 19394 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:31.770843 19394 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
I20260925 00:45:31.771718 19394 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:31.771886 19394 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
W20260925 00:45:31.772718 19394 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 00:45:31.883423 19398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:31.883726 19398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:31.883808 19398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:31.888582 19398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:31.888669 19398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:31.888760 19398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:45:31.895041 19398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.72.84:50655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:45313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=34325
--tserver_master_addrs=127.6.72.126:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:31.896206 19398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:31.897289 19398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:31.908761 19406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:31.908900 19403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:31.908773 19404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:31.909133 19398 server_base.cc:1034] running on GCE node
I20260925 00:45:31.909595 19398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:31.910324 19398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:31.911581 19398 hybrid_clock.cc:648] HybridClock initialized: now 1790297131911553 us; error 46 us; skew 500 ppm
Sep 25 00:45:31 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:31.914803 19398 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:45:31.915844 19398 webserver.cc:460] Webserver started at http://127.6.72.66:34325/ using document root <none> and password file <none>
I20260925 00:45:31.916556 19398 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:31.916666 19398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 00:45:31 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:31.940541 19398 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:45:31.945048 19414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:31.946673 19398 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260925 00:45:31.946799 19398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "98af5b6c261440dfae9c306ec84822c2"
format_stamp: "Formatted at 2026-09-25 00:45:30 on dist-test-slave-w8nh"
server_key: "b3d137ae477a5f8b397a87ed1dae3bff"
server_key_iv: "d973e7ba2ebe5cf69b0c23b6a3d49b6e"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:31.947547 19398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:31.991768 19398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:31.995831 19398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:31.996042 19398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:31.996798 19398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:31.997874 19398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:31.997941 19398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:31.997999 19398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:31.998026 19398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:32.009378 19398 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:45313
I20260925 00:45:32.010710 19398 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:45:32.013656 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 19398
I20260925 00:45:32.014138 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.72.84:50655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:46479
--local_ip_for_outbound_sockets=127.6.72.67
--tserver_master_addrs=127.6.72.126:36701
--webserver_port=46865
--webserver_interface=127.6.72.67
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.72.84:50655/kms with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:32.021067 19528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:45313 every 8 connection(s)
Sep 25 00:45:32 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297131, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:32.030301 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:32.014474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:57309 (local address 127.6.72.126:36701)
0925 00:45:32.014626 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:32.014631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:32.014854 (+ 223us) server_negotiation.cc:407] Connection header received
0925 00:45:32.016169 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:32.016174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:32.016233 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:32.016375 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:32.017981 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:32.018819 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:32.020661 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:32.021027 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:32.021124 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:32.024705 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:32.024709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:32.024712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:32.024741 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:32.026660 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:32.027458 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:32.027462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:32.027464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:32.027551 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:32.027970 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:32.027973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:32.027975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:32.028212 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:32.028319 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:32.029978 (+ 1659us) server_negotiation.cc:299] Negotiation successful
0925 00:45:32.030118 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 00:45:32.032354 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:32.013296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36701 (local address 127.6.72.66:57309)
0925 00:45:32.014660 (+ 1364us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:32.014730 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:32.015940 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:32.016720 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:32.016730 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:32.017091 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:32.017759 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:32.017778 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:32.019032 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:32.019037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:32.020498 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:32.020509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:32.021347 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:32.021352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:32.021539 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:32.022258 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:32.022286 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:32.024506 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:32.026949 (+ 2443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:32.026956 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:32.026972 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:32.027334 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:32.027672 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:32.027675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:32.027677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:32.027851 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:32.029347 (+ 1496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:32.029354 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:32.029691 (+ 337us) client_negotiation.cc:769] Sending connection context
0925 00:45:32.030860 (+ 1169us) client_negotiation.cc:240] Negotiation successful
0925 00:45:32.031198 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1129,"thread_start_us":135,"threads_started":1}
I20260925 00:45:32.033754 19529 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36701
I20260925 00:45:32.033989 19529 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:32.034627 19529 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
W20260925 00:45:32.036419 19529 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:36701 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 00:45:32.036548 19529 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:32.036724 19529 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
I20260925 00:45:32.037621 19529 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:32.037791 19529 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
W20260925 00:45:32.038587 19529 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 00:45:32.149262 19532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:32.149531 19532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:32.149580 19532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:32.153904 19532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:32.153962 19532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:32.154072 19532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:45:32.160017 19532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.72.84:50655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:46479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=46865
--tserver_master_addrs=127.6.72.126:36701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:32.161221 19532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:32.162312 19532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:32.173828 19538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:32.173848 19540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:32.173848 19537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:32.174239 19532 server_base.cc:1034] running on GCE node
I20260925 00:45:32.174666 19532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:32.175331 19532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:32.176548 19532 hybrid_clock.cc:648] HybridClock initialized: now 1790297132176509 us; error 57 us; skew 500 ppm
Sep 25 00:45:32 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297132, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:32.179594 19532 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:45:32.180742 19532 webserver.cc:460] Webserver started at http://127.6.72.67:46865/ using document root <none> and password file <none>
I20260925 00:45:32.181324 19532 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:32.181433 19532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 00:45:32 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297132, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:32.207546 19532 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:45:32.210527 19548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:32.211933 19532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:45:32.212095 19532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "2cb93cdcadeb451896dee2098789cfdc"
format_stamp: "Formatted at 2026-09-25 00:45:31 on dist-test-slave-w8nh"
server_key: "bcbdc2b839ba603ceffd763d53529610"
server_key_iv: "85062fb343e02caa9dbbcd905e47f22c"
server_key_version: "kuduclusterkey@0"
I20260925 00:45:32.212738 19532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:32.226949 19532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:32.230432 19532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:32.230657 19532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:32.231395 19532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:32.232604 19532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:32.232695 19532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:32.232766 19532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:32.232808 19532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:32.244150 19532 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:46479
I20260925 00:45:32.244268 19661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:46479 every 8 connection(s)
I20260925 00:45:32.245643 19532 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:45:32.252986 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 19532
Sep 25 00:45:32 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297132, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:32.264992 19666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:32.253787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40100 (local address 127.6.72.126:36701)
0925 00:45:32.254471 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:32.254475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:32.254488 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:45:32.254522 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:32.254525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:32.254569 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:32.254648 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:32.255750 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:32.256634 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:32.257705 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:32.258066 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:32.258228 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:32.260112 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:32.260115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:32.260117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:32.260142 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:32.262298 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:32.262788 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:32.262794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:32.262797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:32.262888 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:32.263692 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:32.263695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:32.263696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:32.263881 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:32.264132 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:32.264634 (+ 502us) server_negotiation.cc:299] Negotiation successful
0925 00:45:32.264768 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":607,"thread_start_us":103,"threads_started":1}
I20260925 00:45:32.266747 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:32.249631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:41863 (local address 127.6.72.126:36701)
0925 00:45:32.249777 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:32.249782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:32.250307 (+ 525us) server_negotiation.cc:407] Connection header received
0925 00:45:32.252004 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:32.252009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:32.252087 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:32.252291 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:32.253824 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:32.254691 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:32.256580 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:32.257091 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:32.257248 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:32.260677 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:32.260682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:32.260686 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:32.260741 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:32.262663 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:32.263730 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:32.263735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:32.263737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:32.263841 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:32.264981 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:32.264984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:32.264986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:32.265301 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:32.265403 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:32.266432 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0925 00:45:32.266567 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":60}
I20260925 00:45:32.267876 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:32.248434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:36701 (local address 127.6.72.67:41863)
0925 00:45:32.250099 (+ 1665us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:32.250158 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:32.251297 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:32.252480 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:32.252488 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:32.252897 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:32.253630 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:32.253654 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:32.254859 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:32.254863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:32.256438 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:32.256449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:32.257430 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:32.257434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:32.257654 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:32.258438 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:32.258461 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:32.260473 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:32.263042 (+ 2569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:32.263050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:32.263293 (+ 243us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:32.263591 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:32.263970 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:32.263976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:32.263980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:32.264185 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:32.265468 (+ 1283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:32.265577 (+ 109us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:32.266117 (+ 540us) client_negotiation.cc:769] Sending connection context
0925 00:45:32.266353 (+ 236us) client_negotiation.cc:240] Negotiation successful
0925 00:45:32.266714 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1427,"thread_start_us":151,"threads_started":1}
I20260925 00:45:32.269282 19662 heartbeater.cc:347] Connected to a master server at 127.6.72.126:36701
I20260925 00:45:32.269556 19662 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:32.270056 19662 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
W20260925 00:45:32.271813 19662 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:36701 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 00:45:32.271981 19662 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:32.272207 19662 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
I20260925 00:45:32.273262 19662 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:32.273474 19662 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
W20260925 00:45:32.274356 19662 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 00:45:32 dist-test-slave-w8nh krb5kdc[18200](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790297132, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:32.772945 19394 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:32.773293 19394 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
I20260925 00:45:32.774477 19196 ts_manager.cc:194] Registered new tserver with Master: 95b572ca25cb41309320414d1219d9b1 (127.6.72.65:40223)
I20260925 00:45:32.776163 19196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:57891
I20260925 00:45:33.038822 19529 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:33.039165 19529 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
I20260925 00:45:33.040122 19196 ts_manager.cc:194] Registered new tserver with Master: 98af5b6c261440dfae9c306ec84822c2 (127.6.72.66:45313)
I20260925 00:45:33.041036 19196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:57309
Sep 25 00:45:33 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297132, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:33.274613 19662 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:33.274912 19662 heartbeater.cc:511] Master 127.6.72.126:36701 requested a full tablet report, sending...
I20260925 00:45:33.275616 19196 ts_manager.cc:194] Registered new tserver with Master: 2cb93cdcadeb451896dee2098789cfdc (127.6.72.67:46479)
I20260925 00:45:33.276418 19196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:41863
I20260925 00:45:33.287478 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:45:33.299019 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.289340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40102 (local address 127.6.72.126:36701)
0925 00:45:33.289608 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:33.289612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:33.289949 (+ 337us) server_negotiation.cc:407] Connection header received
0925 00:45:33.290001 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:33.290004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:33.290059 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:33.290153 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:33.291020 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.291865 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.293044 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.293374 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.293488 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.295168 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:33.295172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:33.295176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:33.295203 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:33.297166 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:33.297633 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:33.297636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:33.297638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:33.297730 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:33.298086 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:33.298089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:33.298091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:33.298309 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:33.298445 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:33.298697 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 00:45:33.298833 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260925 00:45:33.303270 19196 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:45:33.308058 19196 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:45:33.334720 19691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.323825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:40223 (local address 127.0.0.1:44398)
0925 00:45:33.324913 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:33.324925 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:33.324997 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:33.327896 (+ 2899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:33.327903 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:33.327928 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:33.328067 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:33.328076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.329691 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.329695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:33.332004 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:33.332015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.333352 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.333356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:33.333462 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.333504 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 00:45:33.333634 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 00:45:33.333765 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1031,"mutex_wait_us":34,"thread_start_us":79,"threads_started":1}
I20260925 00:45:33.335752 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.324229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44398 (local address 127.6.72.65:40223)
0925 00:45:33.327351 (+ 3122us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:33.327359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:33.327383 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:45:33.327464 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:33.327470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:33.327712 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:33.327877 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:33.328220 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.329471 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.332147 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.333031 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.333139 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.333286 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:33.335480 (+ 2194us) server_negotiation.cc:299] Negotiation successful
0925 00:45:33.335608 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3009,"thread_start_us":67,"threads_started":1}
I20260925 00:45:33.337736 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.321582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:45313 (local address 127.0.0.1:56722)
0925 00:45:33.324282 (+ 2700us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:33.324334 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:33.324474 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:33.329209 (+ 4735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:33.329212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:33.329233 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:33.329327 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:33.329334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.330711 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.330715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:33.332156 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:33.332163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.334230 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.334234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:33.337421 (+ 3187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.337448 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 00:45:33.337535 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 00:45:33.337609 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2503,"mutex_wait_us":312,"thread_start_us":105,"threads_started":1}
I20260925 00:45:33.338268 19690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.322404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.6.72.66:45313)
0925 00:45:33.323968 (+ 1564us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:33.323975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:33.326949 (+ 2974us) server_negotiation.cc:407] Connection header received
0925 00:45:33.328133 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:33.328139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:33.328362 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:33.328502 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:33.329461 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.330559 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.332293 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.333041 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.333139 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.333286 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:33.338010 (+ 4724us) server_negotiation.cc:299] Negotiation successful
0925 00:45:33.338134 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1451,"thread_start_us":124,"threads_started":1}
I20260925 00:45:33.339265 19328 tablet_service.cc:1467] Processing CreateTablet for tablet 00ccbeae7eba460c97dd7103fd1e0881 (DEFAULT_TABLE table=test-table [id=6df7c5559e8c4e6a9796ca61c093bf9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:45:33.341291 19328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ccbeae7eba460c97dd7103fd1e0881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:33.341409 19462 tablet_service.cc:1467] Processing CreateTablet for tablet 00ccbeae7eba460c97dd7103fd1e0881 (DEFAULT_TABLE table=test-table [id=6df7c5559e8c4e6a9796ca61c093bf9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:45:33.342490 19462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ccbeae7eba460c97dd7103fd1e0881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:33.342953 19693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.323194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:46479 (local address 127.0.0.1:57660)
0925 00:45:33.324636 (+ 1442us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:33.324650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:33.324739 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:33.338647 (+ 13908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:33.338650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:33.338671 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:33.338768 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:33.338774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.340364 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.340367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:33.341491 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:33.341502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.342582 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.342586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:33.342681 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.342706 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 00:45:33.342756 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 00:45:33.342824 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1381,"thread_start_us":98,"threads_started":1}
I20260925 00:45:33.343565 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.323434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57660 (local address 127.6.72.67:46479)
0925 00:45:33.329652 (+ 6218us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:33.338140 (+ 8488us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:33.338172 (+ 32us) server_negotiation.cc:407] Connection header received
0925 00:45:33.338259 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:33.338264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:33.338479 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:33.338926 (+ 447us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:33.339093 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.340175 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.341655 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.342439 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.343142 (+ 703us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.343287 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:33.343340 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 00:45:33.343428 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6094,"thread_start_us":77,"threads_started":1}
I20260925 00:45:33.346370 19596 tablet_service.cc:1467] Processing CreateTablet for tablet 00ccbeae7eba460c97dd7103fd1e0881 (DEFAULT_TABLE table=test-table [id=6df7c5559e8c4e6a9796ca61c093bf9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:45:33.347383 19596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ccbeae7eba460c97dd7103fd1e0881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:33.355885 19698 tablet_bootstrap.cc:492] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: Bootstrap starting.
I20260925 00:45:33.355906 19696 tablet_bootstrap.cc:492] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: Bootstrap starting.
I20260925 00:45:33.358388 19698 tablet_bootstrap.cc:654] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:33.359984 19696 tablet_bootstrap.cc:654] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:33.360355 19698 log.cc:824] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:33.360863 19696 log.cc:824] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:33.362946 19698 tablet_bootstrap.cc:492] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: No bootstrap required, opened a new log
I20260925 00:45:33.363155 19698 ts_tablet_manager.cc:1397] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260925 00:45:33.364151 19697 tablet_bootstrap.cc:492] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1: Bootstrap starting.
Sep 25 00:45:33 dist-test-slave-w8nh krb5kdc[18200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297132, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for HTTP/127.6.72.84@KRBTEST.COM
I20260925 00:45:33.367244 19697 tablet_bootstrap.cc:654] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:33.367511 19698 raft_consensus.cc:348] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 } } peers { permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 } } peers { permanent_uuid: "95b572ca25cb41309320414d1219d9b1" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 40223 } }
I20260925 00:45:33.367918 19698 raft_consensus.cc:374] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:33.368017 19698 raft_consensus.cc:729] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb93cdcadeb451896dee2098789cfdc, State: Initialized, Role: FOLLOWER
I20260925 00:45:33.368168 19697 log.cc:824] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:33.368692 19698 consensus_queue.cc:260] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 } } peers { permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 } } peers { permanent_uuid: "95b572ca25cb41309320414d1219d9b1" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 40223 } }
I20260925 00:45:33.369848 19698 ts_tablet_manager.cc:1428] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260925 00:45:33.370270 19662 heartbeater.cc:503] Master 127.6.72.126:36701 was elected leader, sending a full tablet report...
I20260925 00:45:33.374989 19697 tablet_bootstrap.cc:492] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1: No bootstrap required, opened a new log
I20260925 00:45:33.375206 19697 ts_tablet_manager.cc:1397] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260925 00:45:33.375463 19696 tablet_bootstrap.cc:492] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: No bootstrap required, opened a new log
I20260925 00:45:33.375656 19696 ts_tablet_manager.cc:1397] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.002s
I20260925 00:45:33.379463 19697 raft_consensus.cc:348] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 } } peers { permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 } } peers { permanent_uuid: "95b572ca25cb41309320414d1219d9b1" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 40223 } }
I20260925 00:45:33.379491 19696 raft_consensus.cc:348] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 } } peers { permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 } } peers { permanent_uuid: "95b572ca25cb41309320414d1219d9b1" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 40223 } }
I20260925 00:45:33.379729 19697 raft_consensus.cc:374] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:33.379735 19696 raft_consensus.cc:374] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:33.379783 19697 raft_consensus.cc:729] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b572ca25cb41309320414d1219d9b1, State: Initialized, Role: FOLLOWER
I20260925 00:45:33.379786 19696 raft_consensus.cc:729] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98af5b6c261440dfae9c306ec84822c2, State: Initialized, Role: FOLLOWER
I20260925 00:45:33.380293 19696 consensus_queue.cc:260] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 } } peers { permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 } } peers { permanent_uuid: "95b572ca25cb41309320414d1219d9b1" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 40223 } }
I20260925 00:45:33.380538 19697 consensus_queue.cc:260] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 } } peers { permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 } } peers { permanent_uuid: "95b572ca25cb41309320414d1219d9b1" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 40223 } }
I20260925 00:45:33.381364 19696 ts_tablet_manager.cc:1428] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260925 00:45:33.381722 19529 heartbeater.cc:503] Master 127.6.72.126:36701 was elected leader, sending a full tablet report...
I20260925 00:45:33.381569 19697 ts_tablet_manager.cc:1428] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:45:33.383332 19394 heartbeater.cc:503] Master 127.6.72.126:36701 was elected leader, sending a full tablet report...
I20260925 00:45:33.484586 19704 raft_consensus.cc:482] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:45:33.484833 19704 raft_consensus.cc:504] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 } } peers { permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 } } peers { permanent_uuid: "95b572ca25cb41309320414d1219d9b1" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 40223 } }
I20260925 00:45:33.486301 19704 leader_election.cc:288] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98af5b6c261440dfae9c306ec84822c2 (127.6.72.66:45313), 2cb93cdcadeb451896dee2098789cfdc (127.6.72.67:46479)
W20260925 00:45:33.495478 19395 tablet.cc:2369] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:45:33.502883 19663 tablet.cc:2369] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:45:33.505034 19690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.487933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:50645 (local address 127.6.72.66:45313)
0925 00:45:33.488362 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:33.488368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:33.494363 (+ 5995us) server_negotiation.cc:407] Connection header received
0925 00:45:33.494613 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:33.494618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:33.494697 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:33.494753 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:33.497245 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.498018 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.502415 (+ 4397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.503015 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.503110 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.503193 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:33.504774 (+ 1581us) server_negotiation.cc:299] Negotiation successful
0925 00:45:33.504872 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292}
I20260925 00:45:33.505755 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.486783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:45313 (local address 127.6.72.65:50645)
0925 00:45:33.494248 (+ 7465us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:33.494267 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:33.494459 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:33.494962 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:33.494965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:33.494986 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:33.495159 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:33.495168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.498611 (+ 3443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.498615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:33.499951 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:33.499964 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.503305 (+ 3341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.503309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:33.503424 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.503452 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 00:45:33.505475 (+ 2023us) client_negotiation.cc:240] Negotiation successful
0925 00:45:33.505570 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7377,"thread_start_us":146,"threads_started":1}
I20260925 00:45:33.506314 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.488136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:56039 (local address 127.6.72.67:46479)
0925 00:45:33.488476 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:33.488481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:33.496466 (+ 7985us) server_negotiation.cc:407] Connection header received
0925 00:45:33.496694 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:33.496698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:33.496756 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:33.496809 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:33.500145 (+ 3336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.500903 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.503597 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.504185 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.504305 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.504378 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:33.506076 (+ 1698us) server_negotiation.cc:299] Negotiation successful
0925 00:45:33.506188 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260925 00:45:33.507014 19706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.487271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:46479 (local address 127.6.72.65:56039)
0925 00:45:33.496363 (+ 9092us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:33.496379 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:33.496540 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:33.498182 (+ 1642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:33.498185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:33.498327 (+ 142us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:33.498494 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:33.498503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.501090 (+ 2587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.501094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:33.502241 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:33.502253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.504486 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.504490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:33.504601 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.504628 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 00:45:33.506766 (+ 2138us) client_negotiation.cc:240] Negotiation successful
0925 00:45:33.506849 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9022,"thread_start_us":103,"threads_started":1}
I20260925 00:45:33.507778 19483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ccbeae7eba460c97dd7103fd1e0881" candidate_uuid: "95b572ca25cb41309320414d1219d9b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98af5b6c261440dfae9c306ec84822c2" is_pre_election: true
I20260925 00:45:33.508118 19483 raft_consensus.cc:2454] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95b572ca25cb41309320414d1219d9b1 in term 0.
I20260925 00:45:33.508433 19612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ccbeae7eba460c97dd7103fd1e0881" candidate_uuid: "95b572ca25cb41309320414d1219d9b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb93cdcadeb451896dee2098789cfdc" is_pre_election: true
I20260925 00:45:33.508734 19612 raft_consensus.cc:2454] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95b572ca25cb41309320414d1219d9b1 in term 0.
I20260925 00:45:33.508682 19281 leader_election.cc:302] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95b572ca25cb41309320414d1219d9b1, 98af5b6c261440dfae9c306ec84822c2; no voters:
I20260925 00:45:33.509213 19704 raft_consensus.cc:2781] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:45:33.509302 19704 raft_consensus.cc:482] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:45:33.509346 19704 raft_consensus.cc:3037] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:33.510655 19704 raft_consensus.cc:504] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 } } peers { permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 } } peers { permanent_uuid: "95b572ca25cb41309320414d1219d9b1" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 40223 } }
I20260925 00:45:33.511021 19704 leader_election.cc:288] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [CANDIDATE]: Term 1 election: Requested vote from peers 98af5b6c261440dfae9c306ec84822c2 (127.6.72.66:45313), 2cb93cdcadeb451896dee2098789cfdc (127.6.72.67:46479)
I20260925 00:45:33.511415 19612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ccbeae7eba460c97dd7103fd1e0881" candidate_uuid: "95b572ca25cb41309320414d1219d9b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb93cdcadeb451896dee2098789cfdc"
I20260925 00:45:33.511561 19612 raft_consensus.cc:3037] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:33.511691 19483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ccbeae7eba460c97dd7103fd1e0881" candidate_uuid: "95b572ca25cb41309320414d1219d9b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98af5b6c261440dfae9c306ec84822c2"
I20260925 00:45:33.511806 19483 raft_consensus.cc:3037] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:33.512938 19612 raft_consensus.cc:2454] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95b572ca25cb41309320414d1219d9b1 in term 1.
I20260925 00:45:33.513012 19483 raft_consensus.cc:2454] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95b572ca25cb41309320414d1219d9b1 in term 1.
I20260925 00:45:33.513315 19282 leader_election.cc:302] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb93cdcadeb451896dee2098789cfdc, 95b572ca25cb41309320414d1219d9b1; no voters:
I20260925 00:45:33.513501 19704 raft_consensus.cc:2781] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:45:33.513736 19704 raft_consensus.cc:686] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [term 1 LEADER]: Becoming Leader. State: Replica: 95b572ca25cb41309320414d1219d9b1, State: Running, Role: LEADER
I20260925 00:45:33.514087 19704 consensus_queue.cc:237] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 } } peers { permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 } } peers { permanent_uuid: "95b572ca25cb41309320414d1219d9b1" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 40223 } }
W20260925 00:45:33.517684 19530 tablet.cc:2369] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:45:33.518163 19196 catalog_manager.cc:5495] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95b572ca25cb41309320414d1219d9b1 (127.6.72.65). New cstate: current_term: 1 leader_uuid: "95b572ca25cb41309320414d1219d9b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95b572ca25cb41309320414d1219d9b1" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 40223 } health_report { overall_health: HEALTHY } } }
I20260925 00:45:33.573410 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:33.567935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.6.72.65:40223)
0925 00:45:33.568098 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:33.568103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:33.568120 (+ 17us) server_negotiation.cc:407] Connection header received
0925 00:45:33.568208 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:33.568212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:33.568308 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:33.568424 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:45:33.568788 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.569594 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.571099 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:33.571436 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:33.571545 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:33.571986 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:45:33.572603 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:45:33.572722 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:33.573174 (+ 452us) server_negotiation.cc:299] Negotiation successful
0925 00:45:33.573241 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 00:45:33.591768 19612 raft_consensus.cc:1261] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc [term 1 FOLLOWER]: Refusing update from remote peer 95b572ca25cb41309320414d1219d9b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:45:33.592146 19483 raft_consensus.cc:1261] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2 [term 1 FOLLOWER]: Refusing update from remote peer 95b572ca25cb41309320414d1219d9b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:45:33.592880 19708 consensus_queue.cc:1035] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98af5b6c261440dfae9c306ec84822c2" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:45:33.593417 19708 consensus_queue.cc:1035] T 00ccbeae7eba460c97dd7103fd1e0881 P 95b572ca25cb41309320414d1219d9b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb93cdcadeb451896dee2098789cfdc" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:45:33.614693 19712 mvcc.cc:205] Tried to move back new op lower bound from 7333057059182915584 to 7333057058876542976. Current Snapshot: MvccSnapshot[applied={T|T < 7333057059182915584}]
I20260925 00:45:33.631072 19196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:45:33.631289 19196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:45:33.634706 19196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d415838ca7444881b95c2d6337518420: Sending DeleteTablet for 3 replicas of tablet 00ccbeae7eba460c97dd7103fd1e0881
I20260925 00:45:33.636806 19462 tablet_service.cc:1514] Processing DeleteTablet for tablet 00ccbeae7eba460c97dd7103fd1e0881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:33 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:56722
I20260925 00:45:33.637511 19596 tablet_service.cc:1514] Processing DeleteTablet for tablet 00ccbeae7eba460c97dd7103fd1e0881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:33 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:57660
I20260925 00:45:33.638377 19328 tablet_service.cc:1514] Processing DeleteTablet for tablet 00ccbeae7eba460c97dd7103fd1e0881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:33 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:44398
I20260925 00:45:33.639925 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 19234
I20260925 00:45:33.645879 19713 mvcc.cc:205] Tried to move back new op lower bound from 7333057059182915584 to 7333057058876542976. Current Snapshot: MvccSnapshot[applied={T|T < 7333057059182915584 or (T in {7333057059182915584})}]
I20260925 00:45:33.646549 19720 tablet_replica.cc:330] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: stopping tablet replica
I20260925 00:45:33.647053 19720 raft_consensus.cc:2229] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:45:33.647562 19720 raft_consensus.cc:2258] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:45:33.648602 19721 tablet_replica.cc:330] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: stopping tablet replica
I20260925 00:45:33.648995 19721 raft_consensus.cc:2229] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:45:33.649394 19721 raft_consensus.cc:2258] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:45:33.649432 19720 ts_tablet_manager.cc:1905] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:45:33.652742 19720 ts_tablet_manager.cc:1918] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:45:33.652930 19720 log.cc:1196] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/00ccbeae7eba460c97dd7103fd1e0881
I20260925 00:45:33.653604 19720 ts_tablet_manager.cc:1939] T 00ccbeae7eba460c97dd7103fd1e0881 P 98af5b6c261440dfae9c306ec84822c2: Deleting consensus metadata
I20260925 00:45:33.655231 19180 catalog_manager.cc:4841] TS 98af5b6c261440dfae9c306ec84822c2 (127.6.72.66:45313): tablet 00ccbeae7eba460c97dd7103fd1e0881 (table test-table [id=6df7c5559e8c4e6a9796ca61c093bf9f]) successfully deleted
I20260925 00:45:33.656716 19721 ts_tablet_manager.cc:1905] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:45:33.659554 19721 ts_tablet_manager.cc:1918] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:45:33.659664 19721 log.cc:1196] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/00ccbeae7eba460c97dd7103fd1e0881
I20260925 00:45:33.660037 19721 ts_tablet_manager.cc:1939] T 00ccbeae7eba460c97dd7103fd1e0881 P 2cb93cdcadeb451896dee2098789cfdc: Deleting consensus metadata
I20260925 00:45:33.661549 19181 catalog_manager.cc:4841] TS 2cb93cdcadeb451896dee2098789cfdc (127.6.72.67:46479): tablet 00ccbeae7eba460c97dd7103fd1e0881 (table test-table [id=6df7c5559e8c4e6a9796ca61c093bf9f]) successfully deleted
W20260925 00:45:33.661998 19181 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:40223 (error 108)
W20260925 00:45:33.663141 19693 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:45:33.662790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:40223 (local address 127.0.0.1:44408)
0925 00:45:33.662943 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:33.663036 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:40223: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 00:45:33.663399 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 19398
W20260925 00:45:33.663430 19181 catalog_manager.cc:4584] TS 95b572ca25cb41309320414d1219d9b1 (127.6.72.65:40223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 00ccbeae7eba460c97dd7103fd1e0881: Network error: Client connection negotiation failed: client connection to 127.6.72.65:40223: connect: Connection reset by peer (error 104)
I20260925 00:45:33.671353 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 19532
I20260925 00:45:33.681254 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 19163
I20260925 00:45:33.693212 6433 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 00:45:34.039196 6433 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 00:45:34.039384 6433 mini_ranger.cc:66] Stopping Ranger...
I20260925 00:45:34.385370 6433 mini_ranger.cc:68] Stopped Ranger
2026-09-25 00:45:34.385 UTC [18230] LOG: received smart shutdown request
2026-09-25 00:45:34.389 UTC [18230] LOG: background worker "logical replication launcher" (PID 18239) exited with exit code 1
2026-09-25 00:45:34.389 UTC [18234] LOG: shutting down
2026-09-25 00:45:34.567 UTC [18230] LOG: database system is shut down
2026-09-25T00:45:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49171 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:45:34 dist-test-slave-w8nh krb5kdc[19732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:45:34 dist-test-slave-w8nh krb5kdc[19732](info): set up 2 sockets
Sep 25 00:45:34 dist-test-slave-w8nh krb5kdc[19732](info): commencing operation
krb5kdc: starting...
W20260925 00:45:35.728364 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:45:35 dist-test-slave-w8nh krb5kdc[19732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:45:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:45:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:45:35.859880 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:39699
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:39699,127.6.72.125:45811,127.6.72.124:46079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:35.976156 19745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:35.976526 19745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:35.976610 19745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:35.980856 19745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:45:35.980978 19745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:35.981017 19745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:35.981058 19745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:45:35.981093 19745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:45:35.986863 19745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:39699,127.6.72.125:45811,127.6.72.124:46079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:39699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:35.988132 19745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:35.989300 19745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:35.998091 19753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:35.998098 19750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:35.998085 19751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:35.998339 19745 server_base.cc:1034] running on GCE node
I20260925 00:45:35.999009 19745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:36.000198 19745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:36.001458 19745 hybrid_clock.cc:648] HybridClock initialized: now 1790297136001427 us; error 47 us; skew 500 ppm
Sep 25 00:45:35 dist-test-slave-w8nh krb5kdc[19732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297135, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:36.004302 19745 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:45:36.005304 19745 webserver.cc:460] Webserver started at http://127.6.72.126:45793/ using document root <none> and password file <none>
I20260925 00:45:36.005909 19745 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:36.006006 19745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:36.006273 19745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:45:36.008529 19745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "1dfa1e9716a64c168cab24d3cf104a03"
format_stamp: "Formatted at 2026-09-25 00:45:36 on dist-test-slave-w8nh"
server_key: "84b7edcac1096ed443975903180a497c"
server_key_iv: "ee9caeab836d1c704a9d402ef15702d0"
server_key_version: "encryptionkey@0"
I20260925 00:45:36.009124 19745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "1dfa1e9716a64c168cab24d3cf104a03"
format_stamp: "Formatted at 2026-09-25 00:45:36 on dist-test-slave-w8nh"
server_key: "84b7edcac1096ed443975903180a497c"
server_key_iv: "ee9caeab836d1c704a9d402ef15702d0"
server_key_version: "encryptionkey@0"
I20260925 00:45:36.013114 19745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:45:36.015890 19760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:36.017336 19745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:45:36.017500 19745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "1dfa1e9716a64c168cab24d3cf104a03"
format_stamp: "Formatted at 2026-09-25 00:45:36 on dist-test-slave-w8nh"
server_key: "84b7edcac1096ed443975903180a497c"
server_key_iv: "ee9caeab836d1c704a9d402ef15702d0"
server_key_version: "encryptionkey@0"
I20260925 00:45:36.017635 19745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:36.038389 19745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:36.041599 19745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:36.041826 19745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:36.049155 19745 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:39699
I20260925 00:45:36.049300 19812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:39699 every 8 connection(s)
I20260925 00:45:36.050226 19745 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:45:36.053407 19813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:36.057147 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 19745
I20260925 00:45:36.057273 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:45:36.057555 6433 external_mini_cluster.cc:1428] Setting key ae9dc7e0eb2344fe69bd732932206356
I20260925 00:45:36.057778 19813 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.72.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.125@KRBTEST.COM" created.
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297135, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:36.077739 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.060607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37958 (local address 127.6.72.126:39699)
0925 00:45:36.061184 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.061205 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.061464 (+ 259us) server_negotiation.cc:407] Connection header received
0925 00:45:36.062683 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.062693 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.062997 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.063254 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.064121 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.065227 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.066838 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.067299 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.067447 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.072295 (+ 4848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.072304 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.072324 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.072348 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.074381 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.074895 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.074909 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.074910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.075016 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.075354 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.075358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.075359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.075772 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.075920 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.076319 (+ 399us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.076605 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":143,"threads_started":1}
I20260925 00:45:36.077735 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.060607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:39699 (local address 127.0.0.1:37958)
0925 00:45:36.061283 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.061335 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:36.062193 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:36.063294 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:36.063299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:36.063679 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:36.063953 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.063967 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.065385 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.065391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.066672 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.066689 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.067499 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.067505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.067661 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.068514 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:36.068532 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:36.072056 (+ 3524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:36.074507 (+ 2451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.074517 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.074529 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.074774 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.075123 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.075128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.075130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.075249 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.075899 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:36.075906 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:36.076050 (+ 144us) client_negotiation.cc:769] Sending connection context
0925 00:45:36.076233 (+ 183us) client_negotiation.cc:240] Negotiation successful
0925 00:45:36.076656 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":90,"spinlock_wait_cycles":28544,"thread_start_us":177,"threads_started":1}
I20260925 00:45:36.080545 19813 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:45:36.081468 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.081146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:45811 (local address 127.0.0.1:38182)
0925 00:45:36.081308 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.081384 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260925 00:45:36.081838 19764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.72.125:45811: connect: Connection refused (error 111)
I20260925 00:45:36.082815 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.082579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:45811 (local address 127.0.0.1:38184)
0925 00:45:36.082699 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.082755 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 00:45:36.082984 19813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.125:45811: Network error: Client connection negotiation failed: client connection to 127.6.72.125:45811: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab.
Entry for principal kudu/127.6.72.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab.
I20260925 00:45:36.088129 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.125:45811
--webserver_interface=127.6.72.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab
--principal=kudu/127.6.72.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:39699,127.6.72.125:45811,127.6.72.124:46079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:36.132325 19813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } attempt: 1
I20260925 00:45:36.133420 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.132962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:45811 (local address 127.0.0.1:38198)
0925 00:45:36.133186 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.133291 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260925 00:45:36.134394 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.134119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:45811 (local address 127.0.0.1:38202)
0925 00:45:36.134247 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.134313 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 00:45:36.134609 19813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.125:45811: Network error: Client connection negotiation failed: client connection to 127.6.72.125:45811: connect: Connection refused (error 111)
I20260925 00:45:36.202833 19813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } attempt: 2
I20260925 00:45:36.203814 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.203458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:45811 (local address 127.0.0.1:38208)
0925 00:45:36.203636 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.203716 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 00:45:36.204685 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.204428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:45811 (local address 127.0.0.1:38212)
0925 00:45:36.204552 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.204611 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:45811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260925 00:45:36.204845 19813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.125:45811: Network error: Client connection negotiation failed: client connection to 127.6.72.125:45811: connect: Connection refused (error 111)
W20260925 00:45:36.207343 19820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:36.207623 19820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:36.207710 19820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:36.211918 19820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:45:36.211988 19820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:36.212013 19820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:36.212033 19820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:45:36.212085 19820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:45:36.217775 19820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:39699,127.6.72.125:45811,127.6.72.124:46079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab
--principal=kudu/127.6.72.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.125:45811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.72.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:36.218919 19820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:36.219978 19820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:36.229096 19825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:36.229175 19828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:36.229099 19826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:36.229389 19820 server_base.cc:1034] running on GCE node
I20260925 00:45:36.230135 19820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:36.231299 19820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:36.232507 19820 hybrid_clock.cc:648] HybridClock initialized: now 1790297136232480 us; error 47 us; skew 500 ppm
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297136, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:36.235459 19820 init.cc:377] Logged in from keytab as kudu/127.6.72.125@KRBTEST.COM (short username kudu)
I20260925 00:45:36.236374 19820 webserver.cc:460] Webserver started at http://127.6.72.125:40711/ using document root <none> and password file <none>
I20260925 00:45:36.236908 19820 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:36.237006 19820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:36.237186 19820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:45:36.239218 19820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/data/instance:
uuid: "75523e37b64f428fbba0b695b1bdc3fa"
format_stamp: "Formatted at 2026-09-25 00:45:36 on dist-test-slave-w8nh"
server_key: "aed1a4f7be9d2253dd3f53a2431a3bd5"
server_key_iv: "5fce0eae53748952147f9c6f67ee07e1"
server_key_version: "encryptionkey@0"
I20260925 00:45:36.239756 19820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/wal/instance:
uuid: "75523e37b64f428fbba0b695b1bdc3fa"
format_stamp: "Formatted at 2026-09-25 00:45:36 on dist-test-slave-w8nh"
server_key: "aed1a4f7be9d2253dd3f53a2431a3bd5"
server_key_iv: "5fce0eae53748952147f9c6f67ee07e1"
server_key_version: "encryptionkey@0"
I20260925 00:45:36.243687 19820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260925 00:45:36.246354 19835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:36.247745 19820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:45:36.247861 19820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/wal
uuid: "75523e37b64f428fbba0b695b1bdc3fa"
format_stamp: "Formatted at 2026-09-25 00:45:36 on dist-test-slave-w8nh"
server_key: "aed1a4f7be9d2253dd3f53a2431a3bd5"
server_key_iv: "5fce0eae53748952147f9c6f67ee07e1"
server_key_version: "encryptionkey@0"
I20260925 00:45:36.247973 19820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:36.267764 19820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:36.270807 19820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:36.270964 19820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:36.278383 19820 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.125:45811
I20260925 00:45:36.278412 19887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.125:45811 every 8 connection(s)
I20260925 00:45:36.279379 19820 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
I20260925 00:45:36.282258 19888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:36.285513 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 19820
I20260925 00:45:36.285689 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-1/wal/instance
I20260925 00:45:36.285983 6433 external_mini_cluster.cc:1428] Setting key 84fb8edd94b70879f7157988693011ff
I20260925 00:45:36.286600 19888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.72.124@KRBTEST.COM; defaulting to no policy
I20260925 00:45:36.296007 19813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.124@KRBTEST.COM" created.
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297136, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297135, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
I20260925 00:45:36.304757 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.289049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.6.72.126:39699)
0925 00:45:36.289230 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.289237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.290241 (+ 1004us) server_negotiation.cc:407] Connection header received
0925 00:45:36.291574 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.291580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.291659 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.291785 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.293248 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.294127 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.295914 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.296232 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.296427 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.299368 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.299374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.299380 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.299413 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.301922 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.302597 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.302604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.302607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.302695 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.303133 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.303138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.303140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.303334 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.303457 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.304411 (+ 954us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.304549 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 00:45:36.306057 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.289439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:39699 (local address 127.0.0.1:37960)
0925 00:45:36.290015 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.290093 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:36.291322 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:36.291947 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:36.291954 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:36.292405 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:36.293034 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.293057 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.294269 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.294276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.295769 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.295780 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.296492 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.296499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.296724 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.297413 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:36.297430 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:36.299201 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:36.302072 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.302079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.302097 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.302463 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.302828 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.302834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.302836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.303027 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.303657 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:36.303663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:36.304046 (+ 383us) client_negotiation.cc:769] Sending connection context
0925 00:45:36.304355 (+ 309us) client_negotiation.cc:240] Negotiation successful
0925 00:45:36.304738 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":137,"threads_started":1}
I20260925 00:45:36.307394 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.296597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:45811 (local address 127.0.0.1:38226)
0925 00:45:36.296749 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.296765 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:36.296843 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:36.297642 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:36.297646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:36.297821 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:36.297938 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.297945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.299298 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.299304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.300006 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.300021 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.300744 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.300748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.300863 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.301357 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:36.301378 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:36.302916 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:36.305518 (+ 2602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.305521 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.305523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.305770 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.306137 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.306142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.306145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.306247 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.306844 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:36.306847 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:36.306929 (+ 82us) client_negotiation.cc:769] Sending connection context
0925 00:45:36.307052 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 00:45:36.307204 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 00:45:36.307461 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.296804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38226 (local address 127.6.72.125:45811)
0925 00:45:36.297188 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.297199 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.297225 (+ 26us) server_negotiation.cc:407] Connection header received
0925 00:45:36.297273 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.297279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.297470 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.297610 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.298082 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.299146 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.300153 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.300608 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.300734 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.303064 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.303076 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.303095 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.303129 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.305400 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.305898 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.305907 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.305910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.306014 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.306365 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.306368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.306370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.306668 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.306790 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.307140 (+ 350us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.307288 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":194,"threads_started":1}
I20260925 00:45:36.307922 19888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:45:36.309825 19813 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:45:36.310626 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.310376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35320)
0925 00:45:36.310498 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.310551 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 00:45:36.311327 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.311090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35336)
0925 00:45:36.311223 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.311267 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260925 00:45:36.311514 19813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:46079: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297136, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
I20260925 00:45:36.317785 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.308535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.6.72.125:45811)
0925 00:45:36.308634 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.308639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.308771 (+ 132us) server_negotiation.cc:407] Connection header received
0925 00:45:36.309655 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.309658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.309710 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.309796 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.310232 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.310886 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.311907 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.312157 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.312312 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.314424 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.314428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.314431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.314461 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.315988 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.316525 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.316531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.316532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.316609 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.316938 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.316941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.316942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.317117 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.317230 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.317535 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.317630 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 00:45:36.317785 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.308503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:45811 (local address 127.0.0.1:38242)
0925 00:45:36.308694 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.308705 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:36.308814 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:36.309805 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:36.309808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:36.310003 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:36.310112 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.310118 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.310996 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.310999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.311781 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.311797 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.312313 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.312317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.312407 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.312929 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:36.312944 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:36.314300 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:36.316120 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.316123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.316125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.316429 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.316708 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.316711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.316712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.316824 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.317220 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:36.317223 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:36.317310 (+ 87us) client_negotiation.cc:769] Sending connection context
0925 00:45:36.317425 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 00:45:36.317573 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260925 00:45:36.318760 19888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:45:36.319564 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.319231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35348)
0925 00:45:36.319356 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.319480 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab.
Entry for principal kudu/127.6.72.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab.
W20260925 00:45:36.320402 19836 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
I20260925 00:45:36.320888 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.124:46079
--webserver_interface=127.6.72.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab
--principal=kudu/127.6.72.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:39699,127.6.72.125:45811,127.6.72.124:46079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:36.321530 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.321310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35360)
0925 00:45:36.321421 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.321469 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 00:45:36.321704 19888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:46079: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
I20260925 00:45:36.342707 19813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } attempt: 1
I20260925 00:45:36.343703 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.343323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35362)
0925 00:45:36.343514 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.343597 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260925 00:45:36.344599 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.344340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35372)
0925 00:45:36.344454 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.344522 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 00:45:36.344753 19813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:46079: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
I20260925 00:45:36.371021 19888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } attempt: 1
I20260925 00:45:36.372023 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.371701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35374)
0925 00:45:36.371871 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.371939 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260925 00:45:36.372928 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.372670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35380)
0925 00:45:36.372803 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.372868 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260925 00:45:36.373081 19888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:46079: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
I20260925 00:45:36.419970 19813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } attempt: 2
I20260925 00:45:36.421049 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.420662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35382)
0925 00:45:36.420854 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.420940 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260925 00:45:36.421944 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.421703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35396)
0925 00:45:36.421816 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.421876 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 00:45:36.422145 19813 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:46079: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
W20260925 00:45:36.440400 19895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:36.440745 19895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:36.440834 19895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 00:45:36.441237 19888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } attempt: 2
I20260925 00:45:36.442180 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.441823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35406)
0925 00:45:36.441992 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.442079 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 00:45:36.443029 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.442789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35414)
0925 00:45:36.442907 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.442972 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 00:45:36.443174 19888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:46079: Network error: Client connection negotiation failed: client connection to 127.6.72.124:46079: connect: Connection refused (error 111)
W20260925 00:45:36.446012 19895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:45:36.446125 19895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:36.446164 19895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:36.446226 19895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:45:36.446265 19895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:45:36.452677 19895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:39699,127.6.72.125:45811,127.6.72.124:46079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab
--principal=kudu/127.6.72.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.124:46079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.72.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:36.453912 19895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:36.455015 19895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:36.464105 19901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:36.464095 19903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:36.464088 19900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:36.465449 19895 server_base.cc:1034] running on GCE node
I20260925 00:45:36.466185 19895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:36.467549 19895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:36.468776 19895 hybrid_clock.cc:648] HybridClock initialized: now 1790297136468747 us; error 46 us; skew 500 ppm
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297136, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:36.471841 19895 init.cc:377] Logged in from keytab as kudu/127.6.72.124@KRBTEST.COM (short username kudu)
I20260925 00:45:36.472911 19895 webserver.cc:460] Webserver started at http://127.6.72.124:44819/ using document root <none> and password file <none>
I20260925 00:45:36.473506 19895 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:36.473599 19895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:36.473850 19895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:45:36.476033 19895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/data/instance:
uuid: "3b09dabbc98a4bd099e870eec7bc6390"
format_stamp: "Formatted at 2026-09-25 00:45:36 on dist-test-slave-w8nh"
server_key: "1ba0b53d1fbac7e5088ad98cf3a3ae54"
server_key_iv: "79269e6ad4a69dcf2d64b0ada2479390"
server_key_version: "encryptionkey@0"
I20260925 00:45:36.476692 19895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/wal/instance:
uuid: "3b09dabbc98a4bd099e870eec7bc6390"
format_stamp: "Formatted at 2026-09-25 00:45:36 on dist-test-slave-w8nh"
server_key: "1ba0b53d1fbac7e5088ad98cf3a3ae54"
server_key_iv: "79269e6ad4a69dcf2d64b0ada2479390"
server_key_version: "encryptionkey@0"
I20260925 00:45:36.480867 19895 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:45:36.483564 19910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:36.485029 19895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:45:36.485183 19895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/wal
uuid: "3b09dabbc98a4bd099e870eec7bc6390"
format_stamp: "Formatted at 2026-09-25 00:45:36 on dist-test-slave-w8nh"
server_key: "1ba0b53d1fbac7e5088ad98cf3a3ae54"
server_key_iv: "79269e6ad4a69dcf2d64b0ada2479390"
server_key_version: "encryptionkey@0"
I20260925 00:45:36.485318 19895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:36.497589 19895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:36.500921 19895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:36.501134 19895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:36.508786 19895 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.124:46079
I20260925 00:45:36.508814 19962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.124:46079 every 8 connection(s)
I20260925 00:45:36.509783 19895 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
I20260925 00:45:36.512691 19963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:36.516780 19963 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:45:36.518280 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 19895
I20260925 00:45:36.518420 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790296971355272-6433-0/minicluster-data/master-2/wal/instance
I20260925 00:45:36.518713 6433 external_mini_cluster.cc:1428] Setting key 318a9f173590edcf22a0f3a6d989847e
I20260925 00:45:36.521348 19813 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } attempt: 3
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297135, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297136, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
I20260925 00:45:36.532528 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.521959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35436)
0925 00:45:36.522256 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.522271 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:36.522346 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:36.522886 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:36.522889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:36.523062 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:36.523156 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.523162 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.524112 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.524116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.524675 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.524684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.525474 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.525481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.525560 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.526183 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:36.526217 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:36.527685 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:36.530188 (+ 2503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.530198 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.530201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.530474 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.531052 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.531055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.531057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.531138 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.531890 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:36.531895 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:36.532003 (+ 108us) client_negotiation.cc:769] Sending connection context
0925 00:45:36.532145 (+ 142us) client_negotiation.cc:240] Negotiation successful
0925 00:45:36.532341 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"spinlock_wait_cycles":4480}
I20260925 00:45:36.532703 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.518827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.6.72.126:39699)
0925 00:45:36.518991 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.518996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.520136 (+ 1140us) server_negotiation.cc:407] Connection header received
0925 00:45:36.521452 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.521459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.521522 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.521620 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.522704 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.523406 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.524746 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.525069 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.525166 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.528452 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.528456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.528459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.528482 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.530489 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.531173 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.531179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.531181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.531267 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.531586 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.531590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.531592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.531808 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.531893 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.532397 (+ 504us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.532502 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65,"spinlock_wait_cycles":4352}
I20260925 00:45:36.533730 19966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.519294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:39699 (local address 127.0.0.1:37964)
0925 00:45:36.519963 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.520018 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:36.521150 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:36.521715 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:36.521722 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:36.522170 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:36.522543 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.522564 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.523505 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.523509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.524621 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.524632 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.525219 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.525223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.525460 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.526367 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:36.526386 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:36.528272 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:36.530623 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.530631 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.530640 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.531038 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.531385 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.531389 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.531392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.531475 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.531941 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:36.531947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:36.532105 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 00:45:36.532328 (+ 223us) client_negotiation.cc:240] Negotiation successful
0925 00:45:36.532678 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":42,"thread_start_us":149,"threads_started":1}
I20260925 00:45:36.533735 19968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.522335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.6.72.124:46079)
0925 00:45:36.522666 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.522678 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.522702 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:45:36.522733 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.522735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.522772 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.522857 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.523275 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.524016 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.524803 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.525347 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.525467 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.527869 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.527878 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.527895 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.527919 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.530056 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.530820 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.530827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.530828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.530947 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.531248 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.531251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.531253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.531767 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.531907 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.533228 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.533358 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":267,"thread_start_us":141,"threads_started":1}
I20260925 00:45:36.533743 19967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.520187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35430 (local address 127.6.72.124:46079)
0925 00:45:36.520532 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.520543 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.520583 (+ 40us) server_negotiation.cc:407] Connection header received
0925 00:45:36.521150 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.521174 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.521454 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.521671 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.522632 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.523850 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.524979 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.525373 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.525836 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.528872 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.528876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.528879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.528905 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.531453 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.531945 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.531948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.531949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.532030 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.532362 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.532366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.532367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.532546 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.533359 (+ 813us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.533433 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.533543 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":240,"thread_start_us":150,"threads_started":1}
I20260925 00:45:36.534509 19888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } attempt: 3
I20260925 00:45:36.535247 19963 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:45:36.538467 19813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03: Bootstrap starting.
I20260925 00:45:36.541088 19813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03: Neither blocks nor log segments found. Creating new log.
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297136, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:36.541957 19813 log.cc:824] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:36.545126 19813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03: No bootstrap required, opened a new log
I20260925 00:45:36.545509 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.534933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35442)
0925 00:45:36.535774 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.535789 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:36.535875 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:36.536511 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:36.536517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:36.536715 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:36.536819 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.536826 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.537824 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.537828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.538355 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.538363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.538922 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.538926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.539018 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.539931 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:36.539945 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:36.541588 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:36.543908 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.543912 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.543914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.544152 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.544489 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.544492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.544494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.544583 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.545029 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:36.545032 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:36.545122 (+ 90us) client_negotiation.cc:769] Sending connection context
0925 00:45:36.545209 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 00:45:36.545349 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748}
I20260925 00:45:36.546075 19967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.535908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35442 (local address 127.6.72.124:46079)
0925 00:45:36.536117 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.536121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.536133 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:45:36.536169 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.536172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.536219 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.536364 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.536924 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.537733 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.538473 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.538787 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.539256 (+ 469us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.541731 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.541735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.541738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.541762 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.543785 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.544299 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.544303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.544304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.544384 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.544703 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.544706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.544708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.544915 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.545721 (+ 806us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.545804 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.545928 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260925 00:45:36.547309 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.536420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37972 (local address 127.6.72.126:39699)
0925 00:45:36.536531 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.536535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.536718 (+ 183us) server_negotiation.cc:407] Connection header received
0925 00:45:36.536829 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.536832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.536872 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.536941 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.537786 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.538456 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.539450 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.539681 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.539793 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.542658 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.542662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.542665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.542704 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.545130 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.545627 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.545631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.545633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.545719 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.546042 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.546045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.546047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.546277 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.546433 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.547035 (+ 602us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.547169 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297136, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
I20260925 00:45:36.549655 19888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa: Bootstrap starting.
I20260925 00:45:36.551095 19813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } }
I20260925 00:45:36.551348 19813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:36.551453 19813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfa1e9716a64c168cab24d3cf104a03, State: Initialized, Role: FOLLOWER
I20260925 00:45:36.551959 19813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } }
I20260925 00:45:36.552062 19966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.535740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:45811 (local address 127.0.0.1:38256)
0925 00:45:36.542646 (+ 6906us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.542662 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:36.542757 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:36.543109 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:36.543112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:36.543500 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:36.543629 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.543635 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.544428 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.544434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.545304 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.545315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.546044 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.546047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.546157 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.546607 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:36.546620 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:36.548059 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:36.550219 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.550223 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.550224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.550456 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.550808 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.550812 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.550814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.550916 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.551508 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:36.551511 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:36.551612 (+ 101us) client_negotiation.cc:769] Sending connection context
0925 00:45:36.551764 (+ 152us) client_negotiation.cc:240] Negotiation successful
0925 00:45:36.551901 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279}
I20260925 00:45:36.552090 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.535950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.6.72.125:45811)
0925 00:45:36.536052 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.536056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.542741 (+ 6685us) server_negotiation.cc:407] Connection header received
0925 00:45:36.542844 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.542850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.542899 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.542978 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.543752 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.544314 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.545437 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.545818 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.545897 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.548210 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.548214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.548216 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.548273 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.550098 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.550564 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.550567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.550569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.550654 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.551024 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.551028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.551029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.551304 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.551409 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.551831 (+ 422us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.551952 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 00:45:36.552940 19963 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:45:36.553242 19971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } } }
I20260925 00:45:36.553428 19971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:45:36.553853 19888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:36.554713 19813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:45:36.554845 19888 log.cc:824] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:36.557324 19888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa: No bootstrap required, opened a new log
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
Sep 25 00:45:36 dist-test-slave-w8nh krb5kdc[19732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297136, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
I20260925 00:45:36.562235 19888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } }
I20260925 00:45:36.562482 19888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:36.562554 19888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75523e37b64f428fbba0b695b1bdc3fa, State: Initialized, Role: FOLLOWER
I20260925 00:45:36.563236 19888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } }
I20260925 00:45:36.563669 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.549229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38258 (local address 127.6.72.125:45811)
0925 00:45:36.552490 (+ 3261us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.552496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.552512 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:45:36.552549 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.552554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.552679 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.552793 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.553741 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.554498 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.555600 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.555947 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.556024 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.559771 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.559779 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.559781 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.559813 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.561567 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.562092 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.562096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.562098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.562184 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.562536 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.562540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.562541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.562796 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.562954 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.563345 (+ 391us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.563481 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3205,"thread_start_us":123,"threads_started":1}
I20260925 00:45:36.565140 19888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:45:36.566399 19966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.553477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:46079 (local address 127.0.0.1:35450)
0925 00:45:36.553634 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:36.553650 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:36.553735 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:36.554071 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:36.554074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:36.554237 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:36.554321 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.554328 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.558158 (+ 3830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.558161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.558882 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:36.558895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.559516 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.559519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:36.559616 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.560768 (+ 1152us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:36.560788 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:36.562310 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:36.564784 (+ 2474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.564787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.564789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.565036 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.565310 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:36.565313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:36.565315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:36.565384 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:36.565796 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:36.565799 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:36.565904 (+ 105us) client_negotiation.cc:769] Sending connection context
0925 00:45:36.566031 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 00:45:36.566199 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260925 00:45:36.560842 19745 master_runner.cc:186] Error getting master registration for 127.6.72.125:45811: OK, instance_id { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" instance_seqno: 1790297136276122 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 00:45:36.566756 19745 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 00:45:36.566887 19967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.553588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35450 (local address 127.6.72.124:46079)
0925 00:45:36.553698 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.553701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.553713 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:45:36.553833 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.553835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.553885 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.553978 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.556717 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.558050 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.559012 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.559304 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.559399 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.562449 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.562455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.562457 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.562479 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.564417 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.565143 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.565145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.565149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.565226 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.565465 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.565467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.565469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.565676 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.565778 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.566601 (+ 823us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.566706 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 00:45:36.567593 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:45:36.570319 19963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390: Bootstrap starting.
W20260925 00:45:36.570902 19983 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:45:36.570977 19983 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:45:36.573280 19963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:36.574193 19963 log.cc:824] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:36.576510 19984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } } }
I20260925 00:45:36.576776 19984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:45:36.573536 19820 master_runner.cc:186] Error getting master registration for 127.6.72.124:46079: OK, instance_id { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" instance_seqno: 1790297136506654 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 00:45:36.577855 19820 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 00:45:36.581193 19967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.569993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.6.72.124:46079)
0925 00:45:36.570122 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.570128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.571575 (+ 1447us) server_negotiation.cc:407] Connection header received
0925 00:45:36.571788 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.571791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.571839 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.571889 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.572920 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.573671 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.574875 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.575223 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.575329 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.577374 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.577378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.577381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.577409 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.579186 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.579664 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.579667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.579669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.579751 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.580123 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.580127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.580128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.580409 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.580530 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.580873 (+ 343us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.581013 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 00:45:36.584846 19963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390: No bootstrap required, opened a new log
I20260925 00:45:36.585004 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.571003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.6.72.125:45811)
0925 00:45:36.571119 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.571124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.575388 (+ 4264us) server_negotiation.cc:407] Connection header received
0925 00:45:36.575549 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.575553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.575621 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.575675 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.576764 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.577513 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.578730 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.579068 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.579149 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.580877 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.580881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.580884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.580917 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.582641 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.583493 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.583496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.583498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.583585 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.583889 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.583893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.583895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.584117 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.584319 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.584679 (+ 360us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.584811 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 00:45:36.586483 19995 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:45:36.586639 19995 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:45:36.588204 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:36.570810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37974 (local address 127.6.72.126:39699)
0925 00:45:36.570980 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:36.570984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:36.577938 (+ 6954us) server_negotiation.cc:407] Connection header received
0925 00:45:36.577997 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:36.578004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:36.578067 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:36.578154 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:36.579075 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.579836 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.581028 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:36.581380 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:36.581468 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:36.583179 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:36.583183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:36.583187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:36.583219 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:36.586209 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.586748 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.586752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.586754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.586851 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:36.587171 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:36.587174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:36.587176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:36.587422 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:36.587541 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:36.587915 (+ 374us) server_negotiation.cc:299] Negotiation successful
0925 00:45:36.588045 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260925 00:45:36.591183 19963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } }
I20260925 00:45:36.591414 19963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:36.591482 19963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b09dabbc98a4bd099e870eec7bc6390, State: Initialized, Role: FOLLOWER
I20260925 00:45:36.591940 19963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } }
I20260925 00:45:36.593233 19963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:45:36.600950 20006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } } }
I20260925 00:45:36.601116 20006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:45:36.602795 19895 master_runner.cc:263] No leader master found from master 3b09dabbc98a4bd099e870eec7bc6390
I20260925 00:45:36.602883 19895 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 00:45:36.607820 20017 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:45:36.607916 20017 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 00:45:36.752617 19916 tablet.cc:2369] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:45:36.772573 19841 tablet.cc:2369] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:45:36.793706 19766 tablet.cc:2369] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:45:36.839562 19984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:45:36.839809 19984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } }
I20260925 00:45:36.840947 19984 leader_election.cc:288] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dfa1e9716a64c168cab24d3cf104a03 (127.6.72.126:39699), 3b09dabbc98a4bd099e870eec7bc6390 (127.6.72.124:46079)
I20260925 00:45:36.841667 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75523e37b64f428fbba0b695b1bdc3fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfa1e9716a64c168cab24d3cf104a03" is_pre_election: true
I20260925 00:45:36.841676 19937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75523e37b64f428fbba0b695b1bdc3fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b09dabbc98a4bd099e870eec7bc6390" is_pre_election: true
I20260925 00:45:36.842031 19787 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75523e37b64f428fbba0b695b1bdc3fa in term 0.
I20260925 00:45:36.842053 19937 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75523e37b64f428fbba0b695b1bdc3fa in term 0.
I20260925 00:45:36.842674 19839 leader_election.cc:302] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1dfa1e9716a64c168cab24d3cf104a03, 3b09dabbc98a4bd099e870eec7bc6390, 75523e37b64f428fbba0b695b1bdc3fa; no voters:
I20260925 00:45:36.843004 19984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:45:36.843084 19984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:45:36.843113 19984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:36.844286 19984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } }
I20260925 00:45:36.844672 19984 leader_election.cc:288] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [CANDIDATE]: Term 1 election: Requested vote from peers 1dfa1e9716a64c168cab24d3cf104a03 (127.6.72.126:39699), 3b09dabbc98a4bd099e870eec7bc6390 (127.6.72.124:46079)
I20260925 00:45:36.845057 19937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75523e37b64f428fbba0b695b1bdc3fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b09dabbc98a4bd099e870eec7bc6390"
I20260925 00:45:36.845077 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75523e37b64f428fbba0b695b1bdc3fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfa1e9716a64c168cab24d3cf104a03"
I20260925 00:45:36.845232 19937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:36.845218 19787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:36.846297 19937 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75523e37b64f428fbba0b695b1bdc3fa in term 1.
I20260925 00:45:36.846375 19787 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75523e37b64f428fbba0b695b1bdc3fa in term 1.
I20260925 00:45:36.846632 19836 leader_election.cc:302] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b09dabbc98a4bd099e870eec7bc6390, 75523e37b64f428fbba0b695b1bdc3fa; no voters:
I20260925 00:45:36.846812 19984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:45:36.847018 19984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [term 1 LEADER]: Becoming Leader. State: Replica: 75523e37b64f428fbba0b695b1bdc3fa, State: Running, Role: LEADER
I20260925 00:45:36.847359 19984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } }
I20260925 00:45:36.849363 20019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75523e37b64f428fbba0b695b1bdc3fa. Latest consensus state: current_term: 1 leader_uuid: "75523e37b64f428fbba0b695b1bdc3fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } } }
I20260925 00:45:36.849547 20019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:36.850241 20021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:45:36.851245 20021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:45:36.855916 19937 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [term 1 FOLLOWER]: Refusing update from remote peer 75523e37b64f428fbba0b695b1bdc3fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:45:36.855948 19787 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [term 1 FOLLOWER]: Refusing update from remote peer 75523e37b64f428fbba0b695b1bdc3fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:45:36.856606 20019 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:45:36.856786 19984 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:45:36.861179 20006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75523e37b64f428fbba0b695b1bdc3fa. Latest consensus state: current_term: 1 leader_uuid: "75523e37b64f428fbba0b695b1bdc3fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } } }
I20260925 00:45:36.861351 20006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:45:36.861351 19971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75523e37b64f428fbba0b695b1bdc3fa. Latest consensus state: current_term: 1 leader_uuid: "75523e37b64f428fbba0b695b1bdc3fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } } }
I20260925 00:45:36.861461 19971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:45:36.864671 20019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75523e37b64f428fbba0b695b1bdc3fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } } }
I20260925 00:45:36.864830 20019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:36.864984 20019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75523e37b64f428fbba0b695b1bdc3fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } } }
I20260925 00:45:36.865074 20019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:36.865235 20024 mvcc.cc:205] Tried to move back new op lower bound from 7333057072555044864 to 7333057072529420288. Current Snapshot: MvccSnapshot[applied={T|T < 7333057072555044864}]
I20260925 00:45:36.865260 19971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75523e37b64f428fbba0b695b1bdc3fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } } }
I20260925 00:45:36.865382 19971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:45:36.865576 20006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75523e37b64f428fbba0b695b1bdc3fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dfa1e9716a64c168cab24d3cf104a03" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39699 } } peers { permanent_uuid: "75523e37b64f428fbba0b695b1bdc3fa" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 45811 } } peers { permanent_uuid: "3b09dabbc98a4bd099e870eec7bc6390" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 46079 } } }
I20260925 00:45:36.865681 20006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:45:36.866124 20023 mvcc.cc:205] Tried to move back new op lower bound from 7333057072555044864 to 7333057072529420288. Current Snapshot: MvccSnapshot[applied={T|T < 7333057072555044864}]
I20260925 00:45:36.866434 20021 catalog_manager.cc:1347] Generated new cluster ID: 682c4d74aa484b4d95c17828c1e25a62
I20260925 00:45:36.866497 20021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:45:36.888103 20021 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:45:36.889482 20021 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:45:36.897090 20021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75523e37b64f428fbba0b695b1bdc3fa: Generated new TSK 0
I20260925 00:45:36.898452 20021 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:45:37.572403 19983 catalog_manager.cc:1259] Loaded cluster ID: 682c4d74aa484b4d95c17828c1e25a62
I20260925 00:45:37.572497 19983 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03: loading cluster ID for follower catalog manager: success
I20260925 00:45:37.574234 19983 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03: acquiring CA information for follower catalog manager: success
I20260925 00:45:37.575416 19983 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1dfa1e9716a64c168cab24d3cf104a03: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 00:45:37.609323 20017 catalog_manager.cc:1259] Loaded cluster ID: 682c4d74aa484b4d95c17828c1e25a62
I20260925 00:45:37.609408 20017 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390: loading cluster ID for follower catalog manager: success
I20260925 00:45:37.611076 20017 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390: acquiring CA information for follower catalog manager: success
I20260925 00:45:37.612321 20017 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b09dabbc98a4bd099e870eec7bc6390: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 00:45:37.976890 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 19745
I20260925 00:45:37.983786 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 19820
I20260925 00:45:37.990023 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 19895
2026-09-25T00:45:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3309 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:45:38 dist-test-slave-w8nh krb5kdc[20037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:45:38 dist-test-slave-w8nh krb5kdc[20037](info): set up 2 sockets
Sep 25 00:45:38 dist-test-slave-w8nh krb5kdc[20037](info): commencing operation
krb5kdc: starting...
W20260925 00:45:39.037099 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:45:39 dist-test-slave-w8nh krb5kdc[20037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:45:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:45:39.172324 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:38155
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:38155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:39.286660 20050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:39.287045 20050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:39.287130 20050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:39.291288 20050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:45:39.291399 20050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:39.291438 20050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:39.291498 20050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:45:39.291551 20050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:45:39.297039 20050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:38155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:38155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:39.298207 20050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:39.299270 20050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:39.308101 20055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:39.308301 20058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:39.308344 20050 server_base.cc:1034] running on GCE node
W20260925 00:45:39.308110 20056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:39.309119 20050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:39.310302 20050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:39.311558 20050 hybrid_clock.cc:648] HybridClock initialized: now 1790297139311533 us; error 35 us; skew 500 ppm
Sep 25 00:45:39 dist-test-slave-w8nh krb5kdc[20037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297139, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:39.314656 20050 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:45:39.315639 20050 webserver.cc:460] Webserver started at http://127.6.72.126:46767/ using document root <none> and password file <none>
I20260925 00:45:39.316234 20050 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:39.316382 20050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:39.316627 20050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:45:39.318768 20050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "5d84af20bda6453cb476ae8294a5d51b"
format_stamp: "Formatted at 2026-09-25 00:45:39 on dist-test-slave-w8nh"
server_key: "0f862a4c2a9870a63b4d51b99843f290"
server_key_iv: "46089eaa45de46b21d9a62eb3b95897b"
server_key_version: "encryptionkey@0"
I20260925 00:45:39.319335 20050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "5d84af20bda6453cb476ae8294a5d51b"
format_stamp: "Formatted at 2026-09-25 00:45:39 on dist-test-slave-w8nh"
server_key: "0f862a4c2a9870a63b4d51b99843f290"
server_key_iv: "46089eaa45de46b21d9a62eb3b95897b"
server_key_version: "encryptionkey@0"
I20260925 00:45:39.323163 20050 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:45:39.325706 20065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:39.327065 20050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:45:39.327224 20050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "5d84af20bda6453cb476ae8294a5d51b"
format_stamp: "Formatted at 2026-09-25 00:45:39 on dist-test-slave-w8nh"
server_key: "0f862a4c2a9870a63b4d51b99843f290"
server_key_iv: "46089eaa45de46b21d9a62eb3b95897b"
server_key_version: "encryptionkey@0"
I20260925 00:45:39.327353 20050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:39.339687 20050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:39.343006 20050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:39.343219 20050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:39.350227 20050 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:38155
I20260925 00:45:39.350251 20117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:38155 every 8 connection(s)
I20260925 00:45:39.351279 20050 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:45:39.354013 20118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:39.359161 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 20050
I20260925 00:45:39.359293 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:45:39.359596 6433 external_mini_cluster.cc:1428] Setting key 25ac006600b25a8c11677b93b269d8ba
I20260925 00:45:39.359629 20118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b: Bootstrap starting.
I20260925 00:45:39.362164 20118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:39.362895 20118 log.cc:824] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:39.364825 20118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b: No bootstrap required, opened a new log
Sep 25 00:45:39 dist-test-slave-w8nh krb5kdc[20037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:39.368203 20118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d84af20bda6453cb476ae8294a5d51b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38155 } }
I20260925 00:45:39.368486 20118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:39.368566 20118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d84af20bda6453cb476ae8294a5d51b, State: Initialized, Role: FOLLOWER
I20260925 00:45:39.369026 20118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d84af20bda6453cb476ae8294a5d51b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38155 } }
I20260925 00:45:39.369210 20118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:45:39.369293 20118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:45:39.369400 20118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:39.370471 20118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d84af20bda6453cb476ae8294a5d51b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38155 } }
I20260925 00:45:39.370824 20118 leader_election.cc:302] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d84af20bda6453cb476ae8294a5d51b; no voters:
I20260925 00:45:39.371199 20118 leader_election.cc:288] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:45:39.371275 20123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:45:39.371452 20123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [term 1 LEADER]: Becoming Leader. State: Replica: 5d84af20bda6453cb476ae8294a5d51b, State: Running, Role: LEADER
I20260925 00:45:39.371807 20123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d84af20bda6453cb476ae8294a5d51b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38155 } }
I20260925 00:45:39.372519 20118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:45:39.373531 20124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d84af20bda6453cb476ae8294a5d51b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d84af20bda6453cb476ae8294a5d51b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38155 } } }
I20260925 00:45:39.373670 20124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:39.373955 20125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d84af20bda6453cb476ae8294a5d51b. Latest consensus state: current_term: 1 leader_uuid: "5d84af20bda6453cb476ae8294a5d51b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d84af20bda6453cb476ae8294a5d51b" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 38155 } } }
I20260925 00:45:39.374035 20132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:45:39.374262 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:39.360930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.6.72.126:38155)
0925 00:45:39.361353 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:39.361368 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:39.361401 (+ 33us) server_negotiation.cc:407] Connection header received
0925 00:45:39.362043 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:39.362060 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:39.362440 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:39.362776 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:39.363608 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:39.364632 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:39.365572 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:39.366016 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:39.366188 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:39.368733 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:39.368739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:39.368752 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:39.368778 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:39.370757 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:39.371250 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:39.371258 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:39.371263 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:39.371436 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:39.371775 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:39.371778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:39.371780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:39.372291 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:39.372480 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:39.372922 (+ 442us) server_negotiation.cc:299] Negotiation successful
0925 00:45:39.373246 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":123,"threads_started":1}
I20260925 00:45:39.374106 20125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:39.377264 20132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:45:39.378353 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:45:39.383801 20132 catalog_manager.cc:1347] Generated new cluster ID: 4d890ed9c3b24faeb65a63f041f24d32
I20260925 00:45:39.383883 20132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:45:39.394282 20132 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:45:39.395308 20132 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:45:39.400672 20132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d84af20bda6453cb476ae8294a5d51b: Generated new TSK 0
I20260925 00:45:39.401412 20132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:45:39.435132 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 20050
2026-09-25T00:45:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1443 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:45:39 dist-test-slave-w8nh krb5kdc[20145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:45:39 dist-test-slave-w8nh krb5kdc[20145](info): set up 2 sockets
Sep 25 00:45:39 dist-test-slave-w8nh krb5kdc[20145](info): commencing operation
krb5kdc: starting...
W20260925 00:45:40.472703 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:45:40 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:45:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:45:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:45:40.604077 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41357
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:41357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:40.716617 20158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:40.716892 20158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:40.716969 20158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:40.721149 20158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:45:40.721212 20158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:40.721235 20158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:40.721251 20158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:45:40.721294 20158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:45:40.726734 20158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:41357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:40.727850 20158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:40.728988 20158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:40.737906 20164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:40.737977 20163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:40.737977 20166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:40.738188 20158 server_base.cc:1034] running on GCE node
I20260925 00:45:40.738813 20158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:40.739979 20158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:40.741153 20158 hybrid_clock.cc:648] HybridClock initialized: now 1790297140741131 us; error 38 us; skew 500 ppm
Sep 25 00:45:40 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297140, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:40.747762 20158 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:45:40.748880 20158 webserver.cc:460] Webserver started at http://127.6.72.126:34581/ using document root <none> and password file <none>
I20260925 00:45:40.749518 20158 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:40.749627 20158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:40.749891 20158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:45:40.752166 20158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "be14ac4348764ed78162f2bb11aa7f87"
format_stamp: "Formatted at 2026-09-25 00:45:40 on dist-test-slave-w8nh"
server_key: "60a2d176be6546fcc166403397c333ae"
server_key_iv: "cd73c2c9443d03cb6ebeef2c509dfcc1"
server_key_version: "encryptionkey@0"
I20260925 00:45:40.752900 20158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "be14ac4348764ed78162f2bb11aa7f87"
format_stamp: "Formatted at 2026-09-25 00:45:40 on dist-test-slave-w8nh"
server_key: "60a2d176be6546fcc166403397c333ae"
server_key_iv: "cd73c2c9443d03cb6ebeef2c509dfcc1"
server_key_version: "encryptionkey@0"
I20260925 00:45:40.757153 20158 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:45:40.759613 20173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:40.760848 20158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:45:40.760995 20158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "be14ac4348764ed78162f2bb11aa7f87"
format_stamp: "Formatted at 2026-09-25 00:45:40 on dist-test-slave-w8nh"
server_key: "60a2d176be6546fcc166403397c333ae"
server_key_iv: "cd73c2c9443d03cb6ebeef2c509dfcc1"
server_key_version: "encryptionkey@0"
I20260925 00:45:40.761113 20158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:40.791463 20158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:40.794533 20158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:40.794771 20158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:40.802043 20158 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:41357
I20260925 00:45:40.802057 20225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:41357 every 8 connection(s)
I20260925 00:45:40.802999 20158 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:45:40.805874 20226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:40.811345 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 20158
I20260925 00:45:40.811501 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:45:40.811786 6433 external_mini_cluster.cc:1428] Setting key 4a88fb5c944f6cd6eb4c6a19bde91984
I20260925 00:45:40.811914 20226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Bootstrap starting.
I20260925 00:45:40.814451 20226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:40.815316 20226 log.cc:824] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:40.817570 20226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: No bootstrap required, opened a new log
Sep 25 00:45:40 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:40.822150 20226 raft_consensus.cc:348] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:40.822458 20226 raft_consensus.cc:374] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:40.822533 20226 raft_consensus.cc:729] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be14ac4348764ed78162f2bb11aa7f87, State: Initialized, Role: FOLLOWER
I20260925 00:45:40.823096 20226 consensus_queue.cc:260] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:40.823271 20226 raft_consensus.cc:388] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:45:40.823338 20226 raft_consensus.cc:482] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:45:40.823436 20226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:40.824643 20226 raft_consensus.cc:504] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:40.825021 20226 leader_election.cc:302] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be14ac4348764ed78162f2bb11aa7f87; no voters:
I20260925 00:45:40.825371 20226 leader_election.cc:288] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:45:40.825496 20231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:45:40.825717 20231 raft_consensus.cc:686] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 1 LEADER]: Becoming Leader. State: Replica: be14ac4348764ed78162f2bb11aa7f87, State: Running, Role: LEADER
I20260925 00:45:40.826110 20231 consensus_queue.cc:237] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:40.826723 20226 sys_catalog.cc:564] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:45:40.827497 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:40.813236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.6.72.126:41357)
0925 00:45:40.813647 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:40.813672 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:40.813713 (+ 41us) server_negotiation.cc:407] Connection header received
0925 00:45:40.814350 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:40.814367 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:40.814760 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:40.815038 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:40.815976 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:40.817165 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:40.818306 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:40.818773 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:40.818894 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:40.821751 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:40.821757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:40.821775 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:40.821806 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:40.823878 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:40.824326 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:40.824332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:40.824338 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:40.824487 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:40.824756 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:40.824759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:40.824761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:40.825276 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:40.825474 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:40.826188 (+ 714us) server_negotiation.cc:299] Negotiation successful
0925 00:45:40.826494 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":138,"threads_started":1}
I20260925 00:45:40.827826 20233 sys_catalog.cc:455] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be14ac4348764ed78162f2bb11aa7f87. Latest consensus state: current_term: 1 leader_uuid: "be14ac4348764ed78162f2bb11aa7f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } } }
I20260925 00:45:40.827934 20233 sys_catalog.cc:458] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:40.828272 20232 sys_catalog.cc:455] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be14ac4348764ed78162f2bb11aa7f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } } }
I20260925 00:45:40.828351 20232 sys_catalog.cc:458] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:40.828787 20240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:45:40.831965 20240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:45:40.838456 20240 catalog_manager.cc:1347] Generated new cluster ID: 036254307db1487e952f56ad910b7621
I20260925 00:45:40.838575 20240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:45:40.856038 20240 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:45:40.857084 20240 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:45:40.865283 20240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Generated new TSK 0
I20260925 00:45:40.866072 20240 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:45:40.895119 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41357
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:41.008900 20252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:41.009188 20252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:41.009235 20252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:41.014339 20252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:41.014420 20252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:41.014501 20252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:45:41.020661 20252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:41.021842 20252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:41.022904 20252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:41.033057 20257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:41.033286 20258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:41.033313 20260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:41.033655 20252 server_base.cc:1034] running on GCE node
I20260925 00:45:41.034025 20252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:41.034595 20252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:41.035807 20252 hybrid_clock.cc:648] HybridClock initialized: now 1790297141035785 us; error 34 us; skew 500 ppm
Sep 25 00:45:41 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297141, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:41.038473 20252 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:45:41.039332 20252 webserver.cc:460] Webserver started at http://127.6.72.65:37335/ using document root <none> and password file <none>
I20260925 00:45:41.039908 20252 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:41.039999 20252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:41.040282 20252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:45:41.042349 20252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "53bea28e589e4e2e9516a0352c3f3029"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "80a33da470a0b3d097a6a41b99f70f36"
server_key_iv: "e64fb3c89790195a078999a756f8cc6b"
server_key_version: "encryptionkey@0"
I20260925 00:45:41.042932 20252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "53bea28e589e4e2e9516a0352c3f3029"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "80a33da470a0b3d097a6a41b99f70f36"
server_key_iv: "e64fb3c89790195a078999a756f8cc6b"
server_key_version: "encryptionkey@0"
I20260925 00:45:41.047308 20252 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260925 00:45:41.049777 20267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:41.051040 20252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:45:41.051180 20252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "53bea28e589e4e2e9516a0352c3f3029"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "80a33da470a0b3d097a6a41b99f70f36"
server_key_iv: "e64fb3c89790195a078999a756f8cc6b"
server_key_version: "encryptionkey@0"
I20260925 00:45:41.051297 20252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:41.072113 20252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:41.075166 20252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:41.075383 20252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:41.075999 20252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:41.076963 20252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:41.077036 20252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:41.077111 20252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:41.077152 20252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:41.085949 20252 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:39877
I20260925 00:45:41.085969 20380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:39877 every 8 connection(s)
I20260925 00:45:41.086870 20252 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:45:41.092535 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 20252
I20260925 00:45:41.092679 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:45:41.092964 6433 external_mini_cluster.cc:1428] Setting key aa89178e5a8a99fabd8c8e31b3dd251c
Sep 25 00:45:41 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297141, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:41.102100 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.088763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:43741 (local address 127.6.72.126:41357)
0925 00:45:41.088918 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:41.088923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:41.089988 (+ 1065us) server_negotiation.cc:407] Connection header received
0925 00:45:41.090828 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:41.090831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:41.090876 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:41.090951 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:41.092103 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.093030 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.094590 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.094957 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.095103 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.097956 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:41.097959 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:41.097961 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:41.097985 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:41.099676 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:41.100381 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:41.100384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:41.100385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:41.100484 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:41.100951 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:41.100954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:41.100955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:41.101142 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:41.101226 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:41.101867 (+ 641us) server_negotiation.cc:299] Negotiation successful
0925 00:45:41.101973 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 00:45:41.103334 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.089188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:43741)
0925 00:45:41.089843 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:41.089884 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:41.090627 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:41.091112 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:41.091118 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:41.091461 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:41.091943 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.091956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.093165 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.093168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.094414 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.094427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.095104 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.095107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.095250 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.095927 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:41.095942 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:41.097798 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:41.099797 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:41.099806 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:41.099824 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:41.100282 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:41.100617 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:41.100621 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:41.100626 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:41.100831 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:41.101262 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:41.101271 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:41.101606 (+ 335us) client_negotiation.cc:769] Sending connection context
0925 00:45:41.101835 (+ 229us) client_negotiation.cc:240] Negotiation successful
0925 00:45:41.102209 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:45:41.104676 20381 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41357
I20260925 00:45:41.104952 20381 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:41.105455 20381 heartbeater.cc:511] Master 127.6.72.126:41357 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:45:41.107132 20190 ts_manager.cc:194] Registered new tserver with Master: 53bea28e589e4e2e9516a0352c3f3029 (127.6.72.65:39877)
I20260925 00:45:41.108343 20190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:43741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:45:41.124605 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41357
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:41.237427 20386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:41.237704 20386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:41.237747 20386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:41.241786 20386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:41.241865 20386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:41.241925 20386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:45:41.247169 20386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:41.248188 20386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:41.249225 20386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:41.259152 20394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:41.259155 20391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:41.259155 20392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:41.259532 20386 server_base.cc:1034] running on GCE node
I20260925 00:45:41.259920 20386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:41.260641 20386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:41.261807 20386 hybrid_clock.cc:648] HybridClock initialized: now 1790297141261792 us; error 26 us; skew 500 ppm
Sep 25 00:45:41 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297141, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:41.267663 20386 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:45:41.268761 20386 webserver.cc:460] Webserver started at http://127.6.72.66:43351/ using document root <none> and password file <none>
I20260925 00:45:41.269390 20386 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:41.269490 20386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:41.269936 20386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:45:41.272010 20386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "c933b2c37d69449cba36469dba40606b"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "10f5bbdec9b69045565134b0482245c6"
server_key_iv: "3c6c19362d88576441e96a877634c284"
server_key_version: "encryptionkey@0"
I20260925 00:45:41.272630 20386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "c933b2c37d69449cba36469dba40606b"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "10f5bbdec9b69045565134b0482245c6"
server_key_iv: "3c6c19362d88576441e96a877634c284"
server_key_version: "encryptionkey@0"
I20260925 00:45:41.276819 20386 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:45:41.279402 20401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:41.280808 20386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:45:41.280949 20386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "c933b2c37d69449cba36469dba40606b"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "10f5bbdec9b69045565134b0482245c6"
server_key_iv: "3c6c19362d88576441e96a877634c284"
server_key_version: "encryptionkey@0"
I20260925 00:45:41.281066 20386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:41.299316 20386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:41.302632 20386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:41.302865 20386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:41.303493 20386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:41.304450 20386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:41.304533 20386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:41.304633 20386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:41.304692 20386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:41.320667 20386 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:46339
I20260925 00:45:41.320680 20514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:46339 every 8 connection(s)
I20260925 00:45:41.322150 20386 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:45:41.331893 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 20386
I20260925 00:45:41.332016 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:45:41.332340 6433 external_mini_cluster.cc:1428] Setting key 3adf91f4e39cba6f7c7b1e9a62086fec
Sep 25 00:45:41 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297141, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:41.338619 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.324442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:52637 (local address 127.6.72.126:41357)
0925 00:45:41.324592 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:41.324597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:41.325937 (+ 1340us) server_negotiation.cc:407] Connection header received
0925 00:45:41.326915 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:41.326918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:41.326978 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:41.327057 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:41.328591 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.329600 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.330787 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.331228 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.331353 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.334306 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:41.334310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:41.334312 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:41.334338 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:41.335898 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:41.336627 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:41.336630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:41.336632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:41.336746 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:41.337214 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:41.337218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:41.337221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:41.337452 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:41.337556 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:41.338237 (+ 681us) server_negotiation.cc:299] Negotiation successful
0925 00:45:41.338393 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 00:45:41.339711 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.325016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:52637)
0925 00:45:41.325795 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:41.325836 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:41.326690 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:41.327245 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:41.327256 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:41.327702 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:41.328408 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.328428 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.329734 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.329737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.330668 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.330677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.331458 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.331464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.331677 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.332428 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:41.332449 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:41.334140 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:41.336042 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:41.336048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:41.336064 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:41.336480 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:41.336858 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:41.336861 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:41.336863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:41.337090 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:41.337573 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:41.337582 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:41.337924 (+ 342us) client_negotiation.cc:769] Sending connection context
0925 00:45:41.338143 (+ 219us) client_negotiation.cc:240] Negotiation successful
0925 00:45:41.338523 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":155,"threads_started":1}
I20260925 00:45:41.341343 20515 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41357
I20260925 00:45:41.341616 20515 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:41.342310 20515 heartbeater.cc:511] Master 127.6.72.126:41357 requested a full tablet report, sending...
I20260925 00:45:41.343508 20190 ts_manager.cc:194] Registered new tserver with Master: c933b2c37d69449cba36469dba40606b (127.6.72.66:46339)
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:45:41.344165 20190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:52637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:45:41.368489 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41357
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:41.478155 20520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:41.478432 20520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:41.478477 20520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:41.482437 20520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:41.482487 20520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:41.482546 20520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:45:41.487721 20520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:41.488732 20520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:41.489679 20520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:41.498955 20525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:41.498950 20528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:41.498973 20526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:41.499234 20520 server_base.cc:1034] running on GCE node
I20260925 00:45:41.499650 20520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:41.500372 20520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:41.501531 20520 hybrid_clock.cc:648] HybridClock initialized: now 1790297141501513 us; error 60 us; skew 500 ppm
Sep 25 00:45:41 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297141, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:41.504325 20520 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:45:41.505158 20520 webserver.cc:460] Webserver started at http://127.6.72.67:39831/ using document root <none> and password file <none>
I20260925 00:45:41.505719 20520 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:41.505779 20520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:41.506012 20520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:45:41.508000 20520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "653e16553e2a47908c42b7985d5a3f65"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "9b2c2ee49c80464a173e07db0965d640"
server_key_iv: "c025a0e8b35f29646d8cf61ac92835aa"
server_key_version: "encryptionkey@0"
I20260925 00:45:41.508612 20520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "653e16553e2a47908c42b7985d5a3f65"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "9b2c2ee49c80464a173e07db0965d640"
server_key_iv: "c025a0e8b35f29646d8cf61ac92835aa"
server_key_version: "encryptionkey@0"
I20260925 00:45:41.512437 20520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:45:41.514838 20535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:41.516045 20520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:45:41.516166 20520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "653e16553e2a47908c42b7985d5a3f65"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "9b2c2ee49c80464a173e07db0965d640"
server_key_iv: "c025a0e8b35f29646d8cf61ac92835aa"
server_key_version: "encryptionkey@0"
I20260925 00:45:41.516310 20520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:41.543912 20520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:41.547062 20520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:41.547283 20520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:41.547910 20520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:41.548765 20520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:41.548815 20520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:41.548878 20520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:41.548915 20520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:41.558284 20520 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:33393
I20260925 00:45:41.558311 20648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:33393 every 8 connection(s)
I20260925 00:45:41.559197 20520 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:45:41.564810 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 20520
I20260925 00:45:41.564900 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:45:41.565212 6433 external_mini_cluster.cc:1428] Setting key b10604ceb6aa6c603d142df1234ffc6a
Sep 25 00:45:41 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297141, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:41.572642 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.561193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:42291 (local address 127.6.72.126:41357)
0925 00:45:41.561339 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:41.561343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:41.562366 (+ 1023us) server_negotiation.cc:407] Connection header received
0925 00:45:41.563256 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:41.563259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:41.563317 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:41.563424 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:41.564730 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.565497 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.566633 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.566863 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.566974 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.569311 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:41.569314 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:41.569316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:41.569340 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:41.570759 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:41.571233 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:41.571236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:41.571237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:41.571311 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:41.571658 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:41.571659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:41.571661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:41.571825 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:41.571898 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:41.572417 (+ 519us) server_negotiation.cc:299] Negotiation successful
0925 00:45:41.572518 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:45:41.573576 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.561554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:42291)
0925 00:45:41.562210 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:41.562251 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:41.563051 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:41.563602 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:41.563609 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:41.564048 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:41.564583 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.564597 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.565630 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.565636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.566501 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.566510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.567009 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.567013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.567191 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.567712 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:41.567728 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:41.569165 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:41.570871 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:41.570875 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:41.570894 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:41.571145 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:41.571396 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:41.571399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:41.571400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:41.571577 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:41.571909 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:41.571914 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:41.572143 (+ 229us) client_negotiation.cc:769] Sending connection context
0925 00:45:41.572369 (+ 226us) client_negotiation.cc:240] Negotiation successful
0925 00:45:41.572665 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":116,"threads_started":1}
I20260925 00:45:41.574757 20649 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41357
I20260925 00:45:41.575026 20649 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:41.575609 20649 heartbeater.cc:511] Master 127.6.72.126:41357 requested a full tablet report, sending...
I20260925 00:45:41.576788 20190 ts_manager.cc:194] Registered new tserver with Master: 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393)
I20260925 00:45:41.577715 20190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:42291
I20260925 00:45:41.579034 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:45:41.588235 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.581069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54828 (local address 127.6.72.126:41357)
0925 00:45:41.581190 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:41.581193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:41.581421 (+ 228us) server_negotiation.cc:407] Connection header received
0925 00:45:41.581519 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:41.581521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:41.581556 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:41.581615 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:41.582274 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.582763 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.583550 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.583756 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.583828 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.585219 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:41.585223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:41.585225 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:41.585258 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:41.586431 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:41.586870 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:41.586874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:41.586876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:41.586963 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:41.587243 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:41.587247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:41.587249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:41.587473 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:41.587661 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:41.587906 (+ 245us) server_negotiation.cc:299] Negotiation successful
0925 00:45:41.588057 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 00:45:41.591136 20189 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:45:41.594143 20189 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:45:41.608644 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.602935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:46339 (local address 127.0.0.1:50162)
0925 00:45:41.604076 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:41.604092 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:41.604207 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:41.604752 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:41.604755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:41.604787 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:41.604933 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.604939 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.605974 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.605979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.607124 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.607134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.608029 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.608033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.608154 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.608196 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 00:45:41.608333 (+ 137us) client_negotiation.cc:240] Negotiation successful
0925 00:45:41.608456 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"mutex_wait_us":29,"spinlock_wait_cycles":1792,"thread_start_us":90,"threads_started":1}
I20260925 00:45:41.608654 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.603538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.6.72.66:46339)
0925 00:45:41.603919 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:41.603931 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:41.604200 (+ 269us) server_negotiation.cc:407] Connection header received
0925 00:45:41.604343 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:41.604348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:41.604515 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:41.604638 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:41.605050 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.605781 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.607255 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.607899 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.608009 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.608161 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:41.608395 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 00:45:41.608495 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":100,"threads_started":1}
I20260925 00:45:41.609413 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.602688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33393 (local address 127.0.0.1:55914)
0925 00:45:41.603516 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:41.603566 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:41.603689 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:41.604741 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:41.604744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:41.604764 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:41.604908 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.604916 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.606302 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.606305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.607708 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.607719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.609020 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.609023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.609105 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.609122 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 00:45:41.609213 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 00:45:41.609280 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":113,"spinlock_wait_cycles":2304,"thread_start_us":111,"threads_started":1}
I20260925 00:45:41.609485 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.602813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55914 (local address 127.6.72.67:33393)
0925 00:45:41.604029 (+ 1216us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:41.604040 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:41.604063 (+ 23us) server_negotiation.cc:407] Connection header received
0925 00:45:41.604167 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:41.604176 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:41.604413 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:41.604564 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:41.605050 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.606176 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.607848 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.608703 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.608813 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.608949 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:41.609245 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 00:45:41.609337 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1088,"thread_start_us":132,"threads_started":1}
I20260925 00:45:41.610565 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.602696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:39877 (local address 127.0.0.1:54178)
0925 00:45:41.603947 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:41.603960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:41.604042 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:41.604578 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:41.604584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:41.604606 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:41.604743 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.604751 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.606213 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.606225 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.607708 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.607719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.610154 (+ 2435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.610158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.610260 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.610282 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:45:41.610359 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 00:45:41.610426 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1191,"mutex_wait_us":103,"thread_start_us":76,"threads_started":1}
I20260925 00:45:41.610972 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.603483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54178 (local address 127.6.72.65:39877)
0925 00:45:41.604005 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:41.604017 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:41.604041 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:45:41.604191 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:41.604197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:41.604423 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:41.604582 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:41.604866 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.606078 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.607849 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.610028 (+ 2179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.610132 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.610241 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:41.610733 (+ 492us) server_negotiation.cc:299] Negotiation successful
0925 00:45:41.610838 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":85,"threads_started":1}
I20260925 00:45:41.612142 20442 tablet_service.cc:1467] Processing CreateTablet for tablet be6083c8842d43d6a01945be8f87fcb5 (DEFAULT_TABLE table=test-table [id=00a0da68d8eb4e668c89f2cc0ec8b45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:45:41.612836 20583 tablet_service.cc:1467] Processing CreateTablet for tablet be6083c8842d43d6a01945be8f87fcb5 (DEFAULT_TABLE table=test-table [id=00a0da68d8eb4e668c89f2cc0ec8b45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:45:41.613384 20442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be6083c8842d43d6a01945be8f87fcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:41.613868 20583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be6083c8842d43d6a01945be8f87fcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:41.613662 20315 tablet_service.cc:1467] Processing CreateTablet for tablet be6083c8842d43d6a01945be8f87fcb5 (DEFAULT_TABLE table=test-table [id=00a0da68d8eb4e668c89f2cc0ec8b45a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:45:41.614627 20315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be6083c8842d43d6a01945be8f87fcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:41.620471 20666 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: Bootstrap starting.
I20260925 00:45:41.621039 20667 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Bootstrap starting.
I20260925 00:45:41.624001 20667 tablet_bootstrap.cc:654] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:41.624001 20666 tablet_bootstrap.cc:654] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:41.625041 20667 log.cc:824] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:41.625386 20666 log.cc:824] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:41.625900 20668 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Bootstrap starting.
I20260925 00:45:41.627494 20667 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: No bootstrap required, opened a new log
I20260925 00:45:41.627494 20666 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: No bootstrap required, opened a new log
I20260925 00:45:41.627708 20666 ts_tablet_manager.cc:1397] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 00:45:41.627702 20667 ts_tablet_manager.cc:1397] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260925 00:45:41.631829 20667 raft_consensus.cc:348] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:41.631829 20666 raft_consensus.cc:348] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:41.632143 20666 raft_consensus.cc:374] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:41.632144 20667 raft_consensus.cc:374] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:41.632206 20667 raft_consensus.cc:729] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c933b2c37d69449cba36469dba40606b, State: Initialized, Role: FOLLOWER
I20260925 00:45:41.632215 20666 raft_consensus.cc:729] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 653e16553e2a47908c42b7985d5a3f65, State: Initialized, Role: FOLLOWER
I20260925 00:45:41.632735 20667 consensus_queue.cc:260] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:41.632733 20666 consensus_queue.cc:260] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:41.633450 20649 heartbeater.cc:503] Master 127.6.72.126:41357 was elected leader, sending a full tablet report...
I20260925 00:45:41.633708 20515 heartbeater.cc:503] Master 127.6.72.126:41357 was elected leader, sending a full tablet report...
I20260925 00:45:41.633993 20668 tablet_bootstrap.cc:654] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:41.634646 20667 ts_tablet_manager.cc:1428] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260925 00:45:41.634866 20666 ts_tablet_manager.cc:1428] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260925 00:45:41.635859 20668 log.cc:824] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:41.638605 20668 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: No bootstrap required, opened a new log
I20260925 00:45:41.638823 20668 ts_tablet_manager.cc:1397] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.001s
I20260925 00:45:41.642664 20668 raft_consensus.cc:348] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:41.642931 20668 raft_consensus.cc:374] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:41.642997 20668 raft_consensus.cc:729] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bea28e589e4e2e9516a0352c3f3029, State: Initialized, Role: FOLLOWER
I20260925 00:45:41.643460 20668 consensus_queue.cc:260] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:41.644142 20381 heartbeater.cc:503] Master 127.6.72.126:41357 was elected leader, sending a full tablet report...
I20260925 00:45:41.644513 20668 ts_tablet_manager.cc:1428] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
W20260925 00:45:41.810298 20650 tablet.cc:2369] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:45:41.823559 20516 tablet.cc:2369] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:45:41.838426 20382 tablet.cc:2369] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:45:41.960880 20671 raft_consensus.cc:482] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:45:41.961175 20671 raft_consensus.cc:504] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:41.962421 20671 leader_election.cc:288] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53bea28e589e4e2e9516a0352c3f3029 (127.6.72.65:39877), c933b2c37d69449cba36469dba40606b (127.6.72.66:46339)
I20260925 00:45:41.967877 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.962921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:49243 (local address 127.6.72.65:39877)
0925 00:45:41.963071 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:41.963075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:41.963390 (+ 315us) server_negotiation.cc:407] Connection header received
0925 00:45:41.963493 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:41.963496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:41.963565 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:41.963636 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:41.964027 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.964805 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.966559 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.967216 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.967312 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.967408 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:41.967675 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 00:45:41.967747 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:45:41.967877 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.962832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:35903 (local address 127.6.72.66:46339)
0925 00:45:41.962953 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:41.962958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:41.963260 (+ 302us) server_negotiation.cc:407] Connection header received
0925 00:45:41.963326 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:41.963329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:41.963413 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:41.963499 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:41.963991 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.964805 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.966559 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.967176 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.967301 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.967396 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:41.967662 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 00:45:41.967747 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 00:45:41.967892 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.962731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:46339 (local address 127.6.72.67:35903)
0925 00:45:41.963045 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:41.963060 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:41.963167 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:41.963625 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:41.963629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:41.963645 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:41.963849 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.963861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.964926 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.964930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.966409 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.966417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.967343 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.967347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.967500 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.967532 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 00:45:41.967615 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 00:45:41.967716 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":147}
I20260925 00:45:41.967901 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:41.962824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:39877 (local address 127.6.72.67:49243)
0925 00:45:41.963278 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:41.963293 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:41.963404 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:41.963667 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:41.963670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:41.963687 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:41.963838 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.963845 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.964926 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.964930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.966405 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:41.966417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:41.967343 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:41.967347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:41.967547 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:41.967576 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 00:45:41.967642 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 00:45:41.967716 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":191,"thread_start_us":157,"threads_started":1}
I20260925 00:45:41.968660 20469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6083c8842d43d6a01945be8f87fcb5" candidate_uuid: "653e16553e2a47908c42b7985d5a3f65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c933b2c37d69449cba36469dba40606b" is_pre_election: true
I20260925 00:45:41.968662 20335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6083c8842d43d6a01945be8f87fcb5" candidate_uuid: "653e16553e2a47908c42b7985d5a3f65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53bea28e589e4e2e9516a0352c3f3029" is_pre_election: true
I20260925 00:45:41.968972 20335 raft_consensus.cc:2454] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 653e16553e2a47908c42b7985d5a3f65 in term 0.
I20260925 00:45:41.968991 20469 raft_consensus.cc:2454] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 653e16553e2a47908c42b7985d5a3f65 in term 0.
I20260925 00:45:41.969480 20536 leader_election.cc:302] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 653e16553e2a47908c42b7985d5a3f65, c933b2c37d69449cba36469dba40606b; no voters:
I20260925 00:45:41.969830 20671 raft_consensus.cc:2781] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:45:41.969913 20671 raft_consensus.cc:482] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:45:41.969966 20671 raft_consensus.cc:3037] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:41.971011 20671 raft_consensus.cc:504] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:41.971371 20671 leader_election.cc:288] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [CANDIDATE]: Term 1 election: Requested vote from peers 53bea28e589e4e2e9516a0352c3f3029 (127.6.72.65:39877), c933b2c37d69449cba36469dba40606b (127.6.72.66:46339)
I20260925 00:45:41.971725 20335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6083c8842d43d6a01945be8f87fcb5" candidate_uuid: "653e16553e2a47908c42b7985d5a3f65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53bea28e589e4e2e9516a0352c3f3029"
I20260925 00:45:41.971725 20469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6083c8842d43d6a01945be8f87fcb5" candidate_uuid: "653e16553e2a47908c42b7985d5a3f65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c933b2c37d69449cba36469dba40606b"
I20260925 00:45:41.971853 20335 raft_consensus.cc:3037] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:41.971853 20469 raft_consensus.cc:3037] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:41.973170 20335 raft_consensus.cc:2454] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 653e16553e2a47908c42b7985d5a3f65 in term 1.
I20260925 00:45:41.973170 20469 raft_consensus.cc:2454] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 653e16553e2a47908c42b7985d5a3f65 in term 1.
I20260925 00:45:41.973466 20538 leader_election.cc:302] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53bea28e589e4e2e9516a0352c3f3029, 653e16553e2a47908c42b7985d5a3f65; no voters:
I20260925 00:45:41.973681 20671 raft_consensus.cc:2781] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:45:41.973907 20671 raft_consensus.cc:686] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 1 LEADER]: Becoming Leader. State: Replica: 653e16553e2a47908c42b7985d5a3f65, State: Running, Role: LEADER
I20260925 00:45:41.974268 20671 consensus_queue.cc:237] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:41.977576 20190 catalog_manager.cc:5495] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67). New cstate: current_term: 1 leader_uuid: "653e16553e2a47908c42b7985d5a3f65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } health_report { overall_health: UNKNOWN } } }
I20260925 00:45:42.060837 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.057107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55924 (local address 127.6.72.67:33393)
0925 00:45:42.057257 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:42.057262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:42.057276 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:45:42.057441 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:42.057444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:42.057503 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:42.057614 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:45:42.057885 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.058396 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.059263 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.059481 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.059591 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.059911 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:45:42.060422 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:45:42.060555 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:42.060603 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 00:45:42.060667 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 00:45:42.067005 20335 raft_consensus.cc:1261] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Refusing update from remote peer 653e16553e2a47908c42b7985d5a3f65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:45:42.067005 20469 raft_consensus.cc:1261] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Refusing update from remote peer 653e16553e2a47908c42b7985d5a3f65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:45:42.067791 20671 consensus_queue.cc:1035] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:45:42.067991 20676 consensus_queue.cc:1035] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:45:42.076581 20681 mvcc.cc:205] Tried to move back new op lower bound from 7333057093898690560 to 7333057093529554944. Current Snapshot: MvccSnapshot[applied={T|T < 7333057093898690560}]
I20260925 00:45:42.085829 20189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:45:42.086043 20189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:45:42.088642 20189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Sending DeleteTablet for 3 replicas of tablet be6083c8842d43d6a01945be8f87fcb5
I20260925 00:45:42.089648 20315 tablet_service.cc:1514] Processing DeleteTablet for tablet be6083c8842d43d6a01945be8f87fcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:42 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:54178
I20260925 00:45:42.089824 20583 tablet_service.cc:1514] Processing DeleteTablet for tablet be6083c8842d43d6a01945be8f87fcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:42 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:55914
I20260925 00:45:42.090565 20442 tablet_service.cc:1514] Processing DeleteTablet for tablet be6083c8842d43d6a01945be8f87fcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:42 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:50162
I20260925 00:45:42.090808 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 20252
I20260925 00:45:42.096537 20690 tablet_replica.cc:330] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: stopping tablet replica
I20260925 00:45:42.096818 20691 tablet_replica.cc:330] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: stopping tablet replica
I20260925 00:45:42.097142 20691 raft_consensus.cc:2229] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 00:45:42.097333 20176 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:39877 (error 108)
I20260925 00:45:42.097651 20691 raft_consensus.cc:2258] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:45:42.097939 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 20386
I20260925 00:45:42.099231 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.098176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:39877 (local address 127.0.0.1:54188)
0925 00:45:42.099051 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:42.099136 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":775}
I20260925 00:45:42.099746 20690 raft_consensus.cc:2229] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:45:42.100183 20690 raft_consensus.cc:2258] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 00:45:42.100435 20176 catalog_manager.cc:4584] TS 53bea28e589e4e2e9516a0352c3f3029 (127.6.72.65:39877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be6083c8842d43d6a01945be8f87fcb5: Network error: Client connection negotiation failed: client connection to 127.6.72.65:39877: connect: Connection refused (error 111)
I20260925 00:45:42.101696 20690 ts_tablet_manager.cc:1905] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:45:42.104032 20690 ts_tablet_manager.cc:1918] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:45:42.104148 20690 log.cc:1196] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/be6083c8842d43d6a01945be8f87fcb5
I20260925 00:45:42.104470 20690 ts_tablet_manager.cc:1939] T be6083c8842d43d6a01945be8f87fcb5 P 653e16553e2a47908c42b7985d5a3f65: Deleting consensus metadata
I20260925 00:45:42.105612 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 20520
I20260925 00:45:42.106155 20175 catalog_manager.cc:4841] TS 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393): tablet be6083c8842d43d6a01945be8f87fcb5 (table test-table [id=00a0da68d8eb4e668c89f2cc0ec8b45a]) successfully deleted
I20260925 00:45:42.107159 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.106884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:46339 (local address 127.0.0.1:50170)
0925 00:45:42.107014 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:42.107077 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.66:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 00:45:42.111618 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 20158
I20260925 00:45:42.118216 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41357
--webserver_interface=127.6.72.126
--webserver_port=34581
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:41357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:42.232638 20693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:42.232980 20693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:42.233063 20693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:42.237188 20693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:45:42.237303 20693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:42.237344 20693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:42.237380 20693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:45:42.237416 20693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:45:42.242947 20693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:41357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=34581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:42.244143 20693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:42.245289 20693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:42.258385 20699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:42.258402 20702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:42.258462 20693 server_base.cc:1034] running on GCE node
W20260925 00:45:42.258406 20700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:42.259373 20693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:42.260764 20693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:42.261981 20693 hybrid_clock.cc:648] HybridClock initialized: now 1790297142261943 us; error 51 us; skew 500 ppm
Sep 25 00:45:42 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297142, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:42.265053 20693 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:45:42.266019 20693 webserver.cc:460] Webserver started at http://127.6.72.126:34581/ using document root <none> and password file <none>
I20260925 00:45:42.266638 20693 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:42.266747 20693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:42.270856 20693 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:45:42.273216 20709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:42.274600 20693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:45:42.274762 20693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "be14ac4348764ed78162f2bb11aa7f87"
format_stamp: "Formatted at 2026-09-25 00:45:40 on dist-test-slave-w8nh"
server_key: "60a2d176be6546fcc166403397c333ae"
server_key_iv: "cd73c2c9443d03cb6ebeef2c509dfcc1"
server_key_version: "encryptionkey@0"
I20260925 00:45:42.275408 20693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:42.301419 20693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:42.304659 20693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:42.304908 20693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:42.312636 20693 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:41357
I20260925 00:45:42.312943 20761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:41357 every 8 connection(s)
I20260925 00:45:42.314006 20693 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:45:42.315315 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 20693
I20260925 00:45:42.315824 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:39877
--local_ip_for_outbound_sockets=127.6.72.65
--tserver_master_addrs=127.6.72.126:41357
--webserver_port=37335
--webserver_interface=127.6.72.65
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:42.319620 20762 sys_catalog.cc:263] Verifying existing consensus state
I20260925 00:45:42.321607 20762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Bootstrap starting.
I20260925 00:45:42.338951 20762 log.cc:824] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:42.347985 20762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:45:42.348505 20762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Bootstrap complete.
I20260925 00:45:42.352725 20762 raft_consensus.cc:348] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:42.353197 20762 raft_consensus.cc:729] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: be14ac4348764ed78162f2bb11aa7f87, State: Initialized, Role: FOLLOWER
I20260925 00:45:42.353750 20762 consensus_queue.cc:260] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:42.353904 20762 raft_consensus.cc:388] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:45:42.353996 20762 raft_consensus.cc:482] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:45:42.354111 20762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 1 FOLLOWER]: Advancing to term 2
I20260925 00:45:42.356156 20762 raft_consensus.cc:504] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:42.356541 20762 leader_election.cc:302] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be14ac4348764ed78162f2bb11aa7f87; no voters:
I20260925 00:45:42.356889 20762 leader_election.cc:288] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 00:45:42.356980 20766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 00:45:42.357182 20766 raft_consensus.cc:686] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 2 LEADER]: Becoming Leader. State: Replica: be14ac4348764ed78162f2bb11aa7f87, State: Running, Role: LEADER
I20260925 00:45:42.357483 20766 consensus_queue.cc:237] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:42.358189 20762 sys_catalog.cc:564] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:45:42.358510 20766 sys_catalog.cc:455] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be14ac4348764ed78162f2bb11aa7f87. Latest consensus state: current_term: 2 leader_uuid: "be14ac4348764ed78162f2bb11aa7f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } } }
I20260925 00:45:42.358631 20766 sys_catalog.cc:458] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:42.359006 20770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:45:42.359022 20767 sys_catalog.cc:455] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "be14ac4348764ed78162f2bb11aa7f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } } }
I20260925 00:45:42.359290 20767 sys_catalog.cc:458] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:42.364012 20770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:45:42.364697 20770 catalog_manager.cc:1259] Loaded cluster ID: 036254307db1487e952f56ad910b7621
I20260925 00:45:42.364758 20770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:45:42.366156 20770 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 00:45:42.370961 20770 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 00:45:42.376796 20770 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:45:42.377599 20770 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Loaded TSK: 0
I20260925 00:45:42.378885 20770 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 00:45:42.434880 20764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:42.435209 20764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:42.435288 20764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:42.439388 20764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:42.439503 20764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:42.439604 20764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:45:42.445226 20764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:39877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=37335
--tserver_master_addrs=127.6.72.126:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:42.446401 20764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:42.447477 20764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:42.458719 20789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:42.458811 20790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:42.458719 20792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:42.459232 20764 server_base.cc:1034] running on GCE node
I20260925 00:45:42.459681 20764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:42.460554 20764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:42.461761 20764 hybrid_clock.cc:648] HybridClock initialized: now 1790297142461718 us; error 64 us; skew 500 ppm
Sep 25 00:45:42 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297142, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:42.464895 20764 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:45:42.465884 20764 webserver.cc:460] Webserver started at http://127.6.72.65:37335/ using document root <none> and password file <none>
I20260925 00:45:42.466485 20764 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:42.466598 20764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:42.470829 20764 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:45:42.473335 20799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:42.474823 20764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:45:42.474989 20764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "53bea28e589e4e2e9516a0352c3f3029"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "80a33da470a0b3d097a6a41b99f70f36"
server_key_iv: "e64fb3c89790195a078999a756f8cc6b"
server_key_version: "encryptionkey@0"
I20260925 00:45:42.475590 20764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:42.489215 20764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:42.492959 20764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:42.493222 20764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:42.493934 20764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:42.495357 20806 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 00:45:42.500030 20764 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 00:45:42.500114 20764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260925 00:45:42.500213 20764 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 00:45:42.502180 20764 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 00:45:42.502254 20764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260925 00:45:42.502543 20806 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Bootstrap starting.
I20260925 00:45:42.516347 20764 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:39877
I20260925 00:45:42.516360 20913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:39877 every 8 connection(s)
I20260925 00:45:42.517928 20764 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:45:42.523276 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 20764
I20260925 00:45:42.523908 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:46339
--local_ip_for_outbound_sockets=127.6.72.66
--tserver_master_addrs=127.6.72.126:41357
--webserver_port=43351
--webserver_interface=127.6.72.66
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:42.526994 20806 log.cc:824] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Log is configured to *not* fsync() on all Append() calls
Sep 25 00:45:42 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297142, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:42.535686 20806 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:45:42.536281 20806 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Bootstrap complete.
I20260925 00:45:42.537062 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.520351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:55033 (local address 127.6.72.126:41357)
0925 00:45:42.520923 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:42.520939 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:42.521366 (+ 427us) server_negotiation.cc:407] Connection header received
0925 00:45:42.522504 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:42.522516 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:42.522814 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:42.523139 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:42.524319 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.525367 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.526526 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.526990 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.527166 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.530497 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:42.530503 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:42.530519 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:42.530555 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:42.532900 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.533711 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.533717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.533722 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.533883 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.534387 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.534391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.534392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.534849 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:42.534985 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:42.535806 (+ 821us) server_negotiation.cc:299] Negotiation successful
0925 00:45:42.536128 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":144,"threads_started":1}
I20260925 00:45:42.537287 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.520460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:55033)
0925 00:45:42.521178 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:42.521221 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:42.522076 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:42.523196 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:42.523204 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:42.523642 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:42.524104 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.524119 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.525548 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.525550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.526394 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.526404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.527146 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.527149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.527346 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.527964 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:42.527981 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:42.530236 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:42.533056 (+ 2820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:42.533066 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:42.533086 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:42.533547 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:42.534016 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:42.534023 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:42.534028 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:42.534255 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:42.535004 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:42.535012 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:42.535397 (+ 385us) client_negotiation.cc:769] Sending connection context
0925 00:45:42.535663 (+ 266us) client_negotiation.cc:240] Negotiation successful
0925 00:45:42.536103 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":196,"threads_started":1}
I20260925 00:45:42.537845 20806 ts_tablet_manager.cc:1397] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Time spent bootstrapping tablet: real 0.036s user 0.018s sys 0.016s
I20260925 00:45:42.540446 20914 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41357
I20260925 00:45:42.540724 20914 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:42.540642 20806 raft_consensus.cc:348] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:42.541190 20806 raft_consensus.cc:729] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bea28e589e4e2e9516a0352c3f3029, State: Initialized, Role: FOLLOWER
I20260925 00:45:42.541370 20914 heartbeater.cc:511] Master 127.6.72.126:41357 requested a full tablet report, sending...
I20260925 00:45:42.541692 20806 consensus_queue.cc:260] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:42.542701 20806 ts_tablet_manager.cc:1428] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260925 00:45:42.544469 20726 ts_manager.cc:194] Registered new tserver with Master: 53bea28e589e4e2e9516a0352c3f3029 (127.6.72.65:39877)
I20260925 00:45:42.547765 20726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:55033
I20260925 00:45:42.550770 20914 heartbeater.cc:503] Master 127.6.72.126:41357 was elected leader, sending a full tablet report...
Sep 25 00:45:42 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297142, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.65@KRBTEST.COM
I20260925 00:45:42.558907 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.547405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.6.72.65:39877)
0925 00:45:42.548350 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:42.548365 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:42.548393 (+ 28us) server_negotiation.cc:407] Connection header received
0925 00:45:42.548473 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:42.548479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:42.548708 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:42.549473 (+ 765us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:42.549822 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.550502 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.551601 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.551951 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.552072 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.554751 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:42.554767 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:42.554779 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:42.554802 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:42.556613 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.557292 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.557297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.557298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.557379 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.557723 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.557725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.557726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.557961 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:42.558064 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:42.558634 (+ 570us) server_negotiation.cc:299] Negotiation successful
0925 00:45:42.558776 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":101,"threads_started":1}
I20260925 00:45:42.558991 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.547336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:39877 (local address 127.0.0.1:54194)
0925 00:45:42.547896 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:42.547940 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:42.548063 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:42.548895 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:42.548900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:42.549231 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:42.549337 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.549343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.550631 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.550636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.551482 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.551492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.552078 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.552081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.552203 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.552941 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:42.552952 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:42.554601 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:42.556751 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:42.556763 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:42.556770 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:42.557158 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:42.557485 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:42.557489 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:42.557491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:42.557615 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:42.558086 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:42.558093 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:42.558262 (+ 169us) client_negotiation.cc:769] Sending connection context
0925 00:45:42.558441 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 00:45:42.558761 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":140,"threads_started":1}
I20260925 00:45:42.560274 20848 tablet_service.cc:1514] Processing DeleteTablet for tablet be6083c8842d43d6a01945be8f87fcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:42 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:54194
I20260925 00:45:42.561463 20923 tablet_replica.cc:330] stopping tablet replica
I20260925 00:45:42.561774 20923 raft_consensus.cc:2229] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:45:42.562028 20923 raft_consensus.cc:2258] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:45:42.563030 20923 ts_tablet_manager.cc:1905] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:45:42.566685 20923 ts_tablet_manager.cc:1918] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:45:42.566802 20923 log.cc:1196] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/be6083c8842d43d6a01945be8f87fcb5
I20260925 00:45:42.567070 20923 ts_tablet_manager.cc:1939] T be6083c8842d43d6a01945be8f87fcb5 P 53bea28e589e4e2e9516a0352c3f3029: Deleting consensus metadata
I20260925 00:45:42.568929 20712 catalog_manager.cc:4841] TS 53bea28e589e4e2e9516a0352c3f3029 (127.6.72.65:39877): tablet be6083c8842d43d6a01945be8f87fcb5 (table test-table [id=00a0da68d8eb4e668c89f2cc0ec8b45a]) successfully deleted
W20260925 00:45:42.656695 20918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:42.656978 20918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:42.657028 20918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:42.661331 20918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:42.661405 20918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:42.661477 20918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:45:42.666891 20918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:46339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=43351
--tserver_master_addrs=127.6.72.126:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:42.668016 20918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:42.669103 20918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:42.680572 20931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:42.680572 20929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:42.680821 20918 server_base.cc:1034] running on GCE node
W20260925 00:45:42.680573 20928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:42.681288 20918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:42.682087 20918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:42.683279 20918 hybrid_clock.cc:648] HybridClock initialized: now 1790297142683263 us; error 27 us; skew 500 ppm
Sep 25 00:45:42 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297142, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:42.686280 20918 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:45:42.687285 20918 webserver.cc:460] Webserver started at http://127.6.72.66:43351/ using document root <none> and password file <none>
I20260925 00:45:42.687888 20918 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:42.688000 20918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:42.692137 20918 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:45:42.694419 20938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:42.695808 20918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:45:42.695961 20918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "c933b2c37d69449cba36469dba40606b"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "10f5bbdec9b69045565134b0482245c6"
server_key_iv: "3c6c19362d88576441e96a877634c284"
server_key_version: "encryptionkey@0"
I20260925 00:45:42.696599 20918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:42.706100 20918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:42.709740 20918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:42.710004 20918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:42.710708 20918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:42.711912 20945 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 00:45:42.716435 20918 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 00:45:42.716567 20918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260925 00:45:42.716679 20918 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 00:45:42.718417 20918 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 00:45:42.718467 20918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260925 00:45:42.718761 20945 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Bootstrap starting.
I20260925 00:45:42.729494 20918 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:46339
I20260925 00:45:42.729581 21052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:46339 every 8 connection(s)
I20260925 00:45:42.730541 20918 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:45:42.732964 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 20918
I20260925 00:45:42.733407 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:33393
--local_ip_for_outbound_sockets=127.6.72.67
--tserver_master_addrs=127.6.72.126:41357
--webserver_port=39831
--webserver_interface=127.6.72.67
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:42.737607 20945 log.cc:824] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Log is configured to *not* fsync() on all Append() calls
Sep 25 00:45:42 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297142, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:42.746042 20945 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:45:42.746501 20945 tablet_bootstrap.cc:492] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Bootstrap complete.
I20260925 00:45:42.747829 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.732498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:37911 (local address 127.6.72.126:41357)
0925 00:45:42.732676 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:42.732682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:42.733492 (+ 810us) server_negotiation.cc:407] Connection header received
0925 00:45:42.734890 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:42.734898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:42.734977 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:42.735047 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:42.736923 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.737863 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.739274 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.739732 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.739871 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.743167 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:42.743171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:42.743182 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:42.743215 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:42.745061 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.745780 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.745786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.745791 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.745888 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.746261 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.746268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.746271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.746540 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:42.746640 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:42.747423 (+ 783us) server_negotiation.cc:299] Negotiation successful
0925 00:45:42.747588 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 00:45:42.747905 20945 ts_tablet_manager.cc:1397] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Time spent bootstrapping tablet: real 0.029s user 0.015s sys 0.010s
I20260925 00:45:42.749395 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.732698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:37911)
0925 00:45:42.733288 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:42.733370 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:42.734623 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:42.735524 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:42.735534 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:42.736028 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:42.736734 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.736755 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.738040 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.738045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.739121 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.739130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.739898 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.739902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.740105 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.740892 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:42.740913 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:42.742956 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:42.745195 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:42.745201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:42.745215 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:42.745641 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:42.745990 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:42.745993 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:42.745995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:42.746162 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:42.746695 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:42.746701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:42.747021 (+ 320us) client_negotiation.cc:769] Sending connection context
0925 00:45:42.748079 (+ 1058us) client_negotiation.cc:240] Negotiation successful
0925 00:45:42.748457 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":165,"threads_started":1}
I20260925 00:45:42.750694 21053 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41357
I20260925 00:45:42.750943 21053 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:42.750922 20945 raft_consensus.cc:348] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:42.751420 20945 raft_consensus.cc:729] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c933b2c37d69449cba36469dba40606b, State: Initialized, Role: FOLLOWER
I20260925 00:45:42.751471 21053 heartbeater.cc:511] Master 127.6.72.126:41357 requested a full tablet report, sending...
I20260925 00:45:42.751924 20945 consensus_queue.cc:260] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } }
I20260925 00:45:42.752950 20945 ts_tablet_manager.cc:1428] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Time spent starting tablet: real 0.005s user 0.004s sys 0.004s
I20260925 00:45:42.753860 20726 ts_manager.cc:194] Registered new tserver with Master: c933b2c37d69449cba36469dba40606b (127.6.72.66:46339)
I20260925 00:45:42.755158 20726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:37911
I20260925 00:45:42.757381 21053 heartbeater.cc:503] Master 127.6.72.126:41357 was elected leader, sending a full tablet report...
Sep 25 00:45:42 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297142, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.66@KRBTEST.COM
I20260925 00:45:42.764008 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.754838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:46339 (local address 127.0.0.1:50186)
0925 00:45:42.754954 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:42.754970 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:42.755065 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:42.755739 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:42.755741 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:42.755941 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:42.756033 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.756039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.757035 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.757040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.757689 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.757700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.758410 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.758413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.758520 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.759055 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:42.759073 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:42.760540 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:42.762563 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:42.762566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:42.762567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:42.762765 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:42.763055 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:42.763057 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:42.763059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:42.763129 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:42.763609 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:42.763611 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:42.763692 (+ 81us) client_negotiation.cc:769] Sending connection context
0925 00:45:42.763776 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 00:45:42.763888 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 00:45:42.764210 21059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.754914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50186 (local address 127.6.72.66:46339)
0925 00:45:42.755252 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:42.755265 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:42.755289 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:45:42.755357 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:42.755361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:42.755585 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:42.755740 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:42.756147 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.756920 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.757812 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.758288 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.758397 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.760676 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:42.760691 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:42.760703 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:42.760728 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:42.762455 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.762870 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.762876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.762878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.762966 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.763212 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.763215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.763217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.763502 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:42.763640 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:42.763879 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 00:45:42.764051 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":210,"thread_start_us":140,"threads_started":1}
I20260925 00:45:42.765221 20987 tablet_service.cc:1514] Processing DeleteTablet for tablet be6083c8842d43d6a01945be8f87fcb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:42 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:50186
I20260925 00:45:42.766181 21060 tablet_replica.cc:330] stopping tablet replica
I20260925 00:45:42.766471 21060 raft_consensus.cc:2229] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:45:42.766722 21060 raft_consensus.cc:2258] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:45:42.767730 21060 ts_tablet_manager.cc:1905] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:45:42.771252 21060 ts_tablet_manager.cc:1918] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:45:42.771333 21060 log.cc:1196] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/be6083c8842d43d6a01945be8f87fcb5
I20260925 00:45:42.771625 21060 ts_tablet_manager.cc:1939] T be6083c8842d43d6a01945be8f87fcb5 P c933b2c37d69449cba36469dba40606b: Deleting consensus metadata
I20260925 00:45:42.773276 20710 catalog_manager.cc:4841] TS c933b2c37d69449cba36469dba40606b (127.6.72.66:46339): tablet be6083c8842d43d6a01945be8f87fcb5 (table test-table [id=00a0da68d8eb4e668c89f2cc0ec8b45a]) successfully deleted
W20260925 00:45:42.851998 21056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:42.852353 21056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:42.852432 21056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:42.856736 21056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:42.856820 21056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:42.856909 21056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:45:42.862545 21056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:33393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=39831
--tserver_master_addrs=127.6.72.126:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:42.863672 21056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:42.864799 21056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:42.874388 21068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:42.874388 21066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:42.874534 21056 server_base.cc:1034] running on GCE node
W20260925 00:45:42.874728 21065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:42.875141 21056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:42.875834 21056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:42.877018 21056 hybrid_clock.cc:648] HybridClock initialized: now 1790297142877005 us; error 25 us; skew 500 ppm
Sep 25 00:45:42 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297142, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:42.880331 21056 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:45:42.881314 21056 webserver.cc:460] Webserver started at http://127.6.72.67:39831/ using document root <none> and password file <none>
I20260925 00:45:42.881892 21056 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:42.881964 21056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:42.885792 21056 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260925 00:45:42.887822 21075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:42.889075 21056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:45:42.889219 21056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "653e16553e2a47908c42b7985d5a3f65"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "9b2c2ee49c80464a173e07db0965d640"
server_key_iv: "c025a0e8b35f29646d8cf61ac92835aa"
server_key_version: "encryptionkey@0"
I20260925 00:45:42.889807 21056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:42.902917 21056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:42.905995 21056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:42.906206 21056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:42.906801 21056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:42.907610 21056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:42.907680 21056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:42.907747 21056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:42.907791 21056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:42.917037 21056 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:33393
I20260925 00:45:42.917058 21188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:33393 every 8 connection(s)
I20260925 00:45:42.918066 21056 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:45:42.920339 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 21056
Sep 25 00:45:42 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297142, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:45:42.929059 21192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.920961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.6.72.126:41357)
0925 00:45:42.921198 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:42.921204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:42.921457 (+ 253us) server_negotiation.cc:407] Connection header received
0925 00:45:42.921713 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:42.921715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:42.921761 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:42.921858 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:42.922608 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.923150 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.924018 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.924300 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.924417 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.925837 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:42.925841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:42.925844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:42.925876 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:42.927317 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.927731 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.927735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.927736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.927831 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.928135 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.928139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.928140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.928383 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:42.928605 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:42.928754 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 00:45:42.928879 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260925 00:45:42.932184 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.919948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:57741 (local address 127.6.72.126:41357)
0925 00:45:42.920096 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:42.920101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:42.921608 (+ 1507us) server_negotiation.cc:407] Connection header received
0925 00:45:42.922509 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:42.922512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:42.922556 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:42.922633 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:42.923757 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.924313 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.925432 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.925663 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.925783 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.928403 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:42.928406 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:42.928408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:42.928429 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:42.930136 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.930712 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.930716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.930717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.930801 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:42.931187 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:42.931190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:42.931192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:42.931356 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:42.931455 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:42.931934 (+ 479us) server_negotiation.cc:299] Negotiation successful
0925 00:45:42.932041 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 00:45:42.933108 21191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.920487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:57741)
0925 00:45:42.921434 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:42.921509 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:42.922317 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:42.922801 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:42.922807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:42.923154 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:42.923618 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.923630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.924451 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.924455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.925307 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.925316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.925823 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.925827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.925984 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.926600 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:42.926621 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:42.928200 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:42.930256 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:42.930260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:42.930302 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:42.930609 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:42.930890 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:42.930892 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:42.930894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:42.931077 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:42.931468 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:42.931475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:42.931703 (+ 228us) client_negotiation.cc:769] Sending connection context
0925 00:45:42.931888 (+ 185us) client_negotiation.cc:240] Negotiation successful
0925 00:45:42.932214 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":122,"threads_started":1}
I20260925 00:45:42.934170 21189 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41357
I20260925 00:45:42.934381 21189 heartbeater.cc:464] Registering TS with master...
I20260925 00:45:42.934826 21189 heartbeater.cc:511] Master 127.6.72.126:41357 requested a full tablet report, sending...
I20260925 00:45:42.935734 20726 ts_manager.cc:194] Registered new tserver with Master: 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393)
I20260925 00:45:42.936332 20726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:57741
I20260925 00:45:42.942854 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:45:42.947167 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.943574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54846 (local address 127.6.72.126:41357)
0925 00:45:42.943707 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:42.943713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:42.944049 (+ 336us) server_negotiation.cc:407] Connection header received
0925 00:45:42.944224 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:42.944227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:42.944376 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:42.944458 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:45:42.944674 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.945163 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.945946 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.946148 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.946221 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.946464 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:45:42.946823 (+ 359us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:45:42.946907 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:42.946964 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 00:45:42.947007 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:45:42.947739 20726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:45:42.950987 20726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:45:42.958665 20848 tablet_service.cc:1467] Processing CreateTablet for tablet 6a854d1534d643fbbd0778b5333d4951 (DEFAULT_TABLE table=test-table [id=493aba7b648247b2ba7b3f7f71cf1e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:45:42.958825 20987 tablet_service.cc:1467] Processing CreateTablet for tablet 6a854d1534d643fbbd0778b5333d4951 (DEFAULT_TABLE table=test-table [id=493aba7b648247b2ba7b3f7f71cf1e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:45:42.959329 20848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a854d1534d643fbbd0778b5333d4951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:42.959340 20987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a854d1534d643fbbd0778b5333d4951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:42.962183 20945 tablet_bootstrap.cc:492] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: Bootstrap starting.
I20260925 00:45:42.962476 20806 tablet_bootstrap.cc:492] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Bootstrap starting.
I20260925 00:45:42.962728 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.957807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33393 (local address 127.0.0.1:55934)
0925 00:45:42.958113 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:42.958127 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:42.958202 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:42.959007 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:42.959010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:42.959029 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:42.959137 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.959142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.960631 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.960635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.961410 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:42.961419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.962311 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.962315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:42.962404 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.962457 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 00:45:42.962513 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 00:45:42.962593 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234}
I20260925 00:45:42.963080 20945 tablet_bootstrap.cc:654] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:42.963320 20806 tablet_bootstrap.cc:654] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:42.963688 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:42.958055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55934 (local address 127.6.72.67:33393)
0925 00:45:42.958482 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:42.958499 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:42.958564 (+ 65us) server_negotiation.cc:407] Connection header received
0925 00:45:42.958635 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:42.958638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:42.958811 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:42.959298 (+ 487us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:42.959487 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.960503 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.961537 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:42.962157 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:42.963008 (+ 851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:42.963339 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:42.963405 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 00:45:42.963530 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":194,"threads_started":1}
I20260925 00:45:42.964744 20945 tablet_bootstrap.cc:492] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: No bootstrap required, opened a new log
I20260925 00:45:42.964740 20806 tablet_bootstrap.cc:492] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: No bootstrap required, opened a new log
I20260925 00:45:42.964823 20945 ts_tablet_manager.cc:1397] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260925 00:45:42.964823 20806 ts_tablet_manager.cc:1397] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260925 00:45:42.965245 20945 raft_consensus.cc:348] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:42.965245 20806 raft_consensus.cc:348] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:42.965353 20945 raft_consensus.cc:374] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:42.965353 20806 raft_consensus.cc:374] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:42.965392 20945 raft_consensus.cc:729] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c933b2c37d69449cba36469dba40606b, State: Initialized, Role: FOLLOWER
I20260925 00:45:42.965392 20806 raft_consensus.cc:729] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bea28e589e4e2e9516a0352c3f3029, State: Initialized, Role: FOLLOWER
I20260925 00:45:42.965521 20806 consensus_queue.cc:260] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:42.965526 20945 consensus_queue.cc:260] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:42.965780 20945 ts_tablet_manager.cc:1428] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 00:45:42.965788 20806 ts_tablet_manager.cc:1428] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 00:45:42.967000 21123 tablet_service.cc:1467] Processing CreateTablet for tablet 6a854d1534d643fbbd0778b5333d4951 (DEFAULT_TABLE table=test-table [id=493aba7b648247b2ba7b3f7f71cf1e8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:45:42.968204 21123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a854d1534d643fbbd0778b5333d4951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:45:42.974890 21201 tablet_bootstrap.cc:492] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: Bootstrap starting.
I20260925 00:45:42.977383 21201 tablet_bootstrap.cc:654] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: Neither blocks nor log segments found. Creating new log.
I20260925 00:45:42.978114 21201 log.cc:824] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:42.980172 21201 tablet_bootstrap.cc:492] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: No bootstrap required, opened a new log
I20260925 00:45:42.980424 21201 ts_tablet_manager.cc:1397] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
W20260925 00:45:42.981582 21054 tablet.cc:2369] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:45:42.983620 21201 raft_consensus.cc:348] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:42.983875 21201 raft_consensus.cc:374] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:45:42.983955 21201 raft_consensus.cc:729] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 653e16553e2a47908c42b7985d5a3f65, State: Initialized, Role: FOLLOWER
I20260925 00:45:42.984462 21201 consensus_queue.cc:260] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:42.985139 21189 heartbeater.cc:503] Master 127.6.72.126:41357 was elected leader, sending a full tablet report...
I20260925 00:45:42.985466 21201 ts_tablet_manager.cc:1428] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260925 00:45:43.019349 20915 tablet.cc:2369] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:45:43.024191 20920 raft_consensus.cc:482] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:45:43.024422 20920 raft_consensus.cc:504] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:43.025462 20920 leader_election.cc:288] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393), c933b2c37d69449cba36469dba40606b (127.6.72.66:46339)
I20260925 00:45:43.030275 21059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.025822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:56753 (local address 127.6.72.66:46339)
0925 00:45:43.025964 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:43.025969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:43.026243 (+ 274us) server_negotiation.cc:407] Connection header received
0925 00:45:43.026351 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:43.026354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:43.026416 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:43.026504 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:43.026768 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:43.027531 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:43.028957 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:43.029613 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:43.029706 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:43.029856 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:43.030072 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 00:45:43.030149 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:45:43.030274 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.025829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:41977 (local address 127.6.72.67:33393)
0925 00:45:43.025947 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:43.025952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:43.025979 (+ 27us) server_negotiation.cc:407] Connection header received
0925 00:45:43.026154 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:43.026157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:43.026230 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:43.026310 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:45:43.026611 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:43.027403 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:43.028957 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:43.029551 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:43.029679 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:43.029749 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:43.030072 (+ 323us) server_negotiation.cc:299] Negotiation successful
0925 00:45:43.030149 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 00:45:43.030275 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.025698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33393 (local address 127.6.72.65:41977)
0925 00:45:43.025854 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.025871 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:43.025984 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:43.026341 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:43.026345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:43.026366 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:43.026468 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:43.026472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:43.027505 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:43.027507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:43.028813 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:43.028825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:43.029694 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:43.029698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:43.029834 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:43.029931 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 00:45:43.030006 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 00:45:43.030091 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":57}
I20260925 00:45:43.030293 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.025695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:46339 (local address 127.6.72.65:56753)
0925 00:45:43.026127 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.026138 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:43.026242 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:43.026537 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:43.026542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:43.026563 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:45:43.026663 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:43.026668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:43.027626 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:43.027629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:43.028830 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:43.028836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:43.029723 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:43.029726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:43.029861 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:43.029921 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 00:45:43.030006 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 00:45:43.030091 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":223,"thread_start_us":97,"threads_started":1}
I20260925 00:45:43.030964 21143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a854d1534d643fbbd0778b5333d4951" candidate_uuid: "53bea28e589e4e2e9516a0352c3f3029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "653e16553e2a47908c42b7985d5a3f65" is_pre_election: true
I20260925 00:45:43.031039 21007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a854d1534d643fbbd0778b5333d4951" candidate_uuid: "53bea28e589e4e2e9516a0352c3f3029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c933b2c37d69449cba36469dba40606b" is_pre_election: true
I20260925 00:45:43.031324 21143 raft_consensus.cc:2454] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53bea28e589e4e2e9516a0352c3f3029 in term 0.
I20260925 00:45:43.031353 21007 raft_consensus.cc:2454] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53bea28e589e4e2e9516a0352c3f3029 in term 0.
I20260925 00:45:43.031816 20801 leader_election.cc:302] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53bea28e589e4e2e9516a0352c3f3029, 653e16553e2a47908c42b7985d5a3f65; no voters:
I20260925 00:45:43.032104 20920 raft_consensus.cc:2781] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:45:43.032191 20920 raft_consensus.cc:482] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:45:43.032258 20920 raft_consensus.cc:3037] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:43.033303 20920 raft_consensus.cc:504] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:43.033656 20920 leader_election.cc:288] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 1 election: Requested vote from peers 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393), c933b2c37d69449cba36469dba40606b (127.6.72.66:46339)
I20260925 00:45:43.034018 21143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a854d1534d643fbbd0778b5333d4951" candidate_uuid: "53bea28e589e4e2e9516a0352c3f3029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "653e16553e2a47908c42b7985d5a3f65"
I20260925 00:45:43.034066 21007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a854d1534d643fbbd0778b5333d4951" candidate_uuid: "53bea28e589e4e2e9516a0352c3f3029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c933b2c37d69449cba36469dba40606b"
I20260925 00:45:43.034135 21143 raft_consensus.cc:3037] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:43.034231 21007 raft_consensus.cc:3037] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:45:43.035153 21143 raft_consensus.cc:2454] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53bea28e589e4e2e9516a0352c3f3029 in term 1.
I20260925 00:45:43.035298 21007 raft_consensus.cc:2454] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53bea28e589e4e2e9516a0352c3f3029 in term 1.
I20260925 00:45:43.035422 20801 leader_election.cc:302] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53bea28e589e4e2e9516a0352c3f3029, 653e16553e2a47908c42b7985d5a3f65; no voters:
I20260925 00:45:43.035595 20920 raft_consensus.cc:2781] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:45:43.035809 20920 raft_consensus.cc:686] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 LEADER]: Becoming Leader. State: Replica: 53bea28e589e4e2e9516a0352c3f3029, State: Running, Role: LEADER
I20260925 00:45:43.036083 20920 consensus_queue.cc:237] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:43.039057 20726 catalog_manager.cc:5495] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53bea28e589e4e2e9516a0352c3f3029 (127.6.72.65). New cstate: current_term: 1 leader_uuid: "53bea28e589e4e2e9516a0352c3f3029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } health_report { overall_health: HEALTHY } } }
I20260925 00:45:43.065810 21208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.062129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54206 (local address 127.6.72.65:39877)
0925 00:45:43.062429 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:43.062432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:43.062444 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:45:43.062518 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:43.062520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:43.062573 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:43.062698 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:45:43.062959 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:43.063482 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:43.064348 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:43.064578 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:43.064692 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:43.064962 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:45:43.065399 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:45:43.065531 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:43.065595 (+ 64us) server_negotiation.cc:299] Negotiation successful
0925 00:45:43.065649 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":99,"threads_started":1}
I20260925 00:45:43.069695 21007 raft_consensus.cc:1261] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Refusing update from remote peer 53bea28e589e4e2e9516a0352c3f3029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:45:43.069700 21143 raft_consensus.cc:1261] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65 [term 1 FOLLOWER]: Refusing update from remote peer 53bea28e589e4e2e9516a0352c3f3029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:45:43.070374 20920 consensus_queue.cc:1035] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:45:43.070732 21205 consensus_queue.cc:1035] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:45:43.079836 21210 mvcc.cc:205] Tried to move back new op lower bound from 7333057098006802432 to 7333057097878859776. Current Snapshot: MvccSnapshot[applied={T|T < 7333057098006802432 or (T in {7333057098006802432})}]
I20260925 00:45:43.086758 20725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:45:43.086916 20725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:45:43.089381 20725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Sending DeleteTablet for 3 replicas of tablet 6a854d1534d643fbbd0778b5333d4951
I20260925 00:45:43.090096 20987 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a854d1534d643fbbd0778b5333d4951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:43 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:50186
I20260925 00:45:43.090152 20848 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a854d1534d643fbbd0778b5333d4951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:43 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:54194
I20260925 00:45:43.090174 21123 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a854d1534d643fbbd0778b5333d4951 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:45:43 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:55934
I20260925 00:45:43.090322 21060 tablet_replica.cc:330] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: stopping tablet replica
I20260925 00:45:43.090458 21060 raft_consensus.cc:2229] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:45:43.090521 21218 tablet_replica.cc:330] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: stopping tablet replica
I20260925 00:45:43.090596 21060 raft_consensus.cc:2258] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:45:43.090627 21218 raft_consensus.cc:2229] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:45:43.090950 21218 raft_consensus.cc:2258] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:45:43.091094 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 20764
I20260925 00:45:43.092417 21060 ts_tablet_manager.cc:1905] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:45:43.094529 21060 ts_tablet_manager.cc:1918] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:45:43.094581 21060 log.cc:1196] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/6a854d1534d643fbbd0778b5333d4951
I20260925 00:45:43.094770 21060 ts_tablet_manager.cc:1939] T 6a854d1534d643fbbd0778b5333d4951 P c933b2c37d69449cba36469dba40606b: Deleting consensus metadata
I20260925 00:45:43.095140 21219 tablet_replica.cc:330] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: stopping tablet replica
I20260925 00:45:43.095460 21219 raft_consensus.cc:2229] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:45:43.095834 21219 raft_consensus.cc:2258] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:45:43.095852 20710 catalog_manager.cc:4841] TS c933b2c37d69449cba36469dba40606b (127.6.72.66:46339): tablet 6a854d1534d643fbbd0778b5333d4951 (table test-table [id=493aba7b648247b2ba7b3f7f71cf1e8e]) successfully deleted
I20260925 00:45:43.097149 21219 ts_tablet_manager.cc:1905] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 00:45:43.097678 20712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:39877 (error 108)
I20260925 00:45:43.097915 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 20918
I20260925 00:45:43.100797 21219 ts_tablet_manager.cc:1918] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:45:43.100926 21219 log.cc:1196] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/6a854d1534d643fbbd0778b5333d4951
I20260925 00:45:43.101253 21219 ts_tablet_manager.cc:1939] T 6a854d1534d643fbbd0778b5333d4951 P 653e16553e2a47908c42b7985d5a3f65: Deleting consensus metadata
I20260925 00:45:43.102420 20711 catalog_manager.cc:4841] TS 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393): tablet 6a854d1534d643fbbd0778b5333d4951 (table test-table [id=493aba7b648247b2ba7b3f7f71cf1e8e]) successfully deleted
I20260925 00:45:43.103102 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.102745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:39877 (local address 127.0.0.1:54208)
0925 00:45:43.102906 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.102995 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 00:45:43.103295 20712 catalog_manager.cc:4584] TS 53bea28e589e4e2e9516a0352c3f3029 (127.6.72.65:39877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a854d1534d643fbbd0778b5333d4951: Network error: Client connection negotiation failed: client connection to 127.6.72.65:39877: connect: Connection refused (error 111)
I20260925 00:45:43.104200 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 21056
I20260925 00:45:43.109786 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 20693
I20260925 00:45:43.116708 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41357
--webserver_interface=127.6.72.126
--webserver_port=34581
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:41357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:45:43.230240 21221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:43.230595 21221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:43.230679 21221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:43.234751 21221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:45:43.234851 21221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:43.234894 21221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:43.234956 21221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:45:43.234994 21221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:45:43.240376 21221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:41357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=34581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:43.241467 21221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:43.242496 21221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:43.251008 21226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:43.250990 21229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:43.251030 21227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:43.251127 21221 server_base.cc:1034] running on GCE node
I20260925 00:45:43.251760 21221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:43.252974 21221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:43.254168 21221 hybrid_clock.cc:648] HybridClock initialized: now 1790297143254153 us; error 26 us; skew 500 ppm
Sep 25 00:45:43 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:43.256989 21221 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:45:43.257903 21221 webserver.cc:460] Webserver started at http://127.6.72.126:34581/ using document root <none> and password file <none>
I20260925 00:45:43.258428 21221 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:43.258491 21221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:43.262307 21221 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:45:43.264470 21236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:43.265806 21221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:45:43.265962 21221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "be14ac4348764ed78162f2bb11aa7f87"
format_stamp: "Formatted at 2026-09-25 00:45:40 on dist-test-slave-w8nh"
server_key: "60a2d176be6546fcc166403397c333ae"
server_key_iv: "cd73c2c9443d03cb6ebeef2c509dfcc1"
server_key_version: "encryptionkey@0"
I20260925 00:45:43.266532 21221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:43.307585 21221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:43.310747 21221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:43.310968 21221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:43.318589 21221 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:41357
I20260925 00:45:43.318886 21288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:41357 every 8 connection(s)
I20260925 00:45:43.319779 21221 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:45:43.323078 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 21221
I20260925 00:45:43.323526 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:39877
--local_ip_for_outbound_sockets=127.6.72.65
--tserver_master_addrs=127.6.72.126:41357
--webserver_port=37335
--webserver_interface=127.6.72.65
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:43.324191 21289 sys_catalog.cc:263] Verifying existing consensus state
I20260925 00:45:43.325871 21289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Bootstrap starting.
I20260925 00:45:43.342906 21289 log.cc:824] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:43.353780 21289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:45:43.354161 21289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87: Bootstrap complete.
I20260925 00:45:43.358258 21289 raft_consensus.cc:348] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:43.358687 21289 raft_consensus.cc:729] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: be14ac4348764ed78162f2bb11aa7f87, State: Initialized, Role: FOLLOWER
I20260925 00:45:43.359189 21289 consensus_queue.cc:260] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:43.359339 21289 raft_consensus.cc:388] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:45:43.359432 21289 raft_consensus.cc:482] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:45:43.359560 21289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 2 FOLLOWER]: Advancing to term 3
I20260925 00:45:43.361809 21289 raft_consensus.cc:504] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:43.362212 21289 leader_election.cc:302] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be14ac4348764ed78162f2bb11aa7f87; no voters:
I20260925 00:45:43.362555 21289 leader_election.cc:288] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 00:45:43.362697 21293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 00:45:43.362970 21293 raft_consensus.cc:686] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [term 3 LEADER]: Becoming Leader. State: Replica: be14ac4348764ed78162f2bb11aa7f87, State: Running, Role: LEADER
I20260925 00:45:43.363289 21293 consensus_queue.cc:237] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } }
I20260925 00:45:43.363796 21289 sys_catalog.cc:564] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:45:43.364360 21295 sys_catalog.cc:455] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be14ac4348764ed78162f2bb11aa7f87. Latest consensus state: current_term: 3 leader_uuid: "be14ac4348764ed78162f2bb11aa7f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } } }
I20260925 00:45:43.364480 21295 sys_catalog.cc:458] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:43.364706 21294 sys_catalog.cc:455] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "be14ac4348764ed78162f2bb11aa7f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be14ac4348764ed78162f2bb11aa7f87" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41357 } } }
I20260925 00:45:43.364818 21294 sys_catalog.cc:458] T 00000000000000000000000000000000 P be14ac4348764ed78162f2bb11aa7f87 [sys.catalog]: This master's current role is: LEADER
I20260925 00:45:43.366067 21297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:45:43.370617 21297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:45:43.371166 21297 catalog_manager.cc:1259] Loaded cluster ID: 036254307db1487e952f56ad910b7621
I20260925 00:45:43.371245 21297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 00:45:43.372079 21297 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa7c09cd9ad google::LogMessage::Fail() at ??:0
@ 0x7fa7c09d088a google::LogMessage::SendToLog() at ??:0
@ 0x7fa7c09cd4de google::LogMessage::Flush() at ??:0
@ 0x7fa7c09cea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa7cb30cfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fa7cb30dd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa7cb30a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fa7cb34e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa7c2c1f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa7c2c1fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fa7c2c213f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa7c2c10512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa7c42546db start_thread at ??:0
@ 0x7fa7bf4cb61f clone at ??:0
W20260925 00:45:43.451081 21291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:43.451292 21291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:43.451333 21291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:43.455559 21291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:43.455641 21291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:43.455758 21291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:45:43.461241 21291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:39877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=37335
--tserver_master_addrs=127.6.72.126:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:43.462333 21291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:43.463343 21291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:43.473147 21331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:43.473208 21330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:43.473321 21333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:43.473459 21291 server_base.cc:1034] running on GCE node
I20260925 00:45:43.473908 21291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:43.474704 21291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:43.475920 21291 hybrid_clock.cc:648] HybridClock initialized: now 1790297143475895 us; error 32 us; skew 500 ppm
Sep 25 00:45:43 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:43.479483 21291 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:45:43.480738 21291 webserver.cc:460] Webserver started at http://127.6.72.65:37335/ using document root <none> and password file <none>
I20260925 00:45:43.481468 21291 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:43.481562 21291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:43.486720 21291 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 00:45:43.489563 21340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:43.491230 21291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:45:43.491385 21291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "53bea28e589e4e2e9516a0352c3f3029"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "80a33da470a0b3d097a6a41b99f70f36"
server_key_iv: "e64fb3c89790195a078999a756f8cc6b"
server_key_version: "encryptionkey@0"
I20260925 00:45:43.492115 21291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:43.509740 21291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:43.513736 21291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:43.513957 21291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:43.514786 21291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:43.516222 21347 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 00:45:43.520514 21291 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 00:45:43.520591 21291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260925 00:45:43.520661 21291 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 00:45:43.522861 21291 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 00:45:43.522920 21291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260925 00:45:43.523238 21347 tablet_bootstrap.cc:492] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Bootstrap starting.
I20260925 00:45:43.534116 21291 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:39877
I20260925 00:45:43.534305 21454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:39877 every 8 connection(s)
I20260925 00:45:43.535319 21291 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:45:43.540493 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 21291
I20260925 00:45:43.541043 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:46339
--local_ip_for_outbound_sockets=127.6.72.66
--tserver_master_addrs=127.6.72.126:41357
--webserver_port=43351
--webserver_interface=127.6.72.66
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:43.543382 21347 log.cc:824] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Log is configured to *not* fsync() on all Append() calls
I20260925 00:45:43.552353 21347 tablet_bootstrap.cc:492] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:45:43.552790 21347 tablet_bootstrap.cc:492] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Bootstrap complete.
I20260925 00:45:43.554221 21347 ts_tablet_manager.cc:1397] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Time spent bootstrapping tablet: real 0.031s user 0.027s sys 0.001s
I20260925 00:45:43.557717 21347 raft_consensus.cc:348] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:43.558272 21347 raft_consensus.cc:729] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bea28e589e4e2e9516a0352c3f3029, State: Initialized, Role: FOLLOWER
I20260925 00:45:43.558802 21347 consensus_queue.cc:260] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:43.559862 21347 ts_tablet_manager.cc:1428] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260925 00:45:43.603792 21457 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:45:43.537605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:35491)
0925 00:45:43.538242 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.538307 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:43.539363 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:43.602814 (+ 63451us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":141,"threads_started":1}
W20260925 00:45:43.604388 21455 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:41357 (0 consecutive failures): Network error: Failed to ping master at 127.6.72.126:41357: Client connection negotiation failed: client connection to 127.6.72.126:41357: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 00:45:43.605170 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.604876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:55111)
0925 00:45:43.605020 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.605095 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 00:45:43.605839 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.605611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:59621)
0925 00:45:43.605733 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.605783 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 00:45:43.606045 21455 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 00:45:43.657299 21458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:43.657557 21458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:43.657626 21458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:43.661649 21458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:43.661708 21458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:43.661793 21458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:45:43.666997 21458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:46339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=43351
--tserver_master_addrs=127.6.72.126:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:43.668072 21458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:43.669130 21458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:43.678867 21465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:43.678944 21458 server_base.cc:1034] running on GCE node
W20260925 00:45:43.679047 21466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:43.679108 21468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:43.679605 21458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:43.680351 21458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:43.681586 21458 hybrid_clock.cc:648] HybridClock initialized: now 1790297143681565 us; error 31 us; skew 500 ppm
Sep 25 00:45:43 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:43.684373 21458 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:45:43.685191 21458 webserver.cc:460] Webserver started at http://127.6.72.66:43351/ using document root <none> and password file <none>
I20260925 00:45:43.685729 21458 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:43.685788 21458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:43.689582 21458 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:45:43.691516 21475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:43.692857 21458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:45:43.692989 21458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "c933b2c37d69449cba36469dba40606b"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "10f5bbdec9b69045565134b0482245c6"
server_key_iv: "3c6c19362d88576441e96a877634c284"
server_key_version: "encryptionkey@0"
I20260925 00:45:43.693581 21458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:43.708559 21458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:43.711359 21458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:43.711571 21458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:43.712174 21458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:43.713032 21458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:43.713081 21458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:43.713150 21458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:43.713188 21458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:43.722641 21458 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:46339
I20260925 00:45:43.722667 21588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:46339 every 8 connection(s)
I20260925 00:45:43.723675 21458 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:45:43.727248 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.725754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:36161)
0925 00:45:43.726434 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.726551 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":116,"threads_started":1}
W20260925 00:45:43.727872 21589 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:41357 (0 consecutive failures): Network error: Failed to ping master at 127.6.72.126:41357: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
I20260925 00:45:43.728454 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 21458
I20260925 00:45:43.728667 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.728357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:53717)
0925 00:45:43.728528 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.728601 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 00:45:43.728937 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:33393
--local_ip_for_outbound_sockets=127.6.72.67
--tserver_master_addrs=127.6.72.126:41357
--webserver_port=39831
--webserver_interface=127.6.72.67
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:45:43.729385 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.729108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:41343)
0925 00:45:43.729248 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.729306 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 00:45:43.729570 21589 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 00:45:43.839691 21592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:45:43.839984 21592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:45:43.840030 21592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:45:43.844123 21592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:45:43.844192 21592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:45:43.844317 21592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:45:43.849670 21592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:33393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=39831
--tserver_master_addrs=127.6.72.126:41357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:45:43.850670 21592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:45:43.851648 21592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:45:43.861086 21600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:43.861119 21598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:45:43.861138 21597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:45:43.861374 21592 server_base.cc:1034] running on GCE node
I20260925 00:45:43.861785 21592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:45:43.862406 21592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:45:43.863595 21592 hybrid_clock.cc:648] HybridClock initialized: now 1790297143863577 us; error 29 us; skew 500 ppm
Sep 25 00:45:43 dist-test-slave-w8nh krb5kdc[20145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:45:43.866401 21592 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:45:43.867288 21592 webserver.cc:460] Webserver started at http://127.6.72.67:39831/ using document root <none> and password file <none>
I20260925 00:45:43.867859 21592 fs_manager.cc:359] Metadata directory not provided
I20260925 00:45:43.867951 21592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:45:43.871901 21592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260925 00:45:43.874115 21607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:43.875520 21592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 00:45:43.875641 21592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "653e16553e2a47908c42b7985d5a3f65"
format_stamp: "Formatted at 2026-09-25 00:45:41 on dist-test-slave-w8nh"
server_key: "9b2c2ee49c80464a173e07db0965d640"
server_key_iv: "c025a0e8b35f29646d8cf61ac92835aa"
server_key_version: "encryptionkey@0"
I20260925 00:45:43.876183 21592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:45:43.905717 21592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:45:43.908907 21592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:45:43.909116 21592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:45:43.909736 21592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:45:43.910594 21592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:45:43.910640 21592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:43.910687 21592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:45:43.910743 21592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:45:43.920374 21592 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:33393
I20260925 00:45:43.920392 21720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:33393 every 8 connection(s)
I20260925 00:45:43.921424 21592 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:45:43.925207 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.923631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:54771)
0925 00:45:43.924381 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.924502 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":115,"threads_started":1}
I20260925 00:45:43.925532 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 21592
W20260925 00:45:43.925804 21721 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:41357 (0 consecutive failures): Network error: Failed to ping master at 127.6.72.126:41357: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
I20260925 00:45:43.926502 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.926206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:49239)
0925 00:45:43.926369 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.926434 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260925 00:45:43.927156 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:43.926945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:46247)
0925 00:45:43.927058 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:43.927107 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 00:45:43.927354 21721 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 00:45:44.607161 21725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:44.606696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:47159)
0925 00:45:44.606968 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:44.607044 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
I20260925 00:45:44.730532 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:44.730102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:56155)
0925 00:45:44.730362 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:44.730425 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":84,"threads_started":1}
I20260925 00:45:44.928597 21727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:44.928122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:38999)
0925 00:45:44.928404 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:44.928478 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":108,"threads_started":1}
I20260925 00:45:44.967881 21728 raft_consensus.cc:482] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:45:44.968079 21728 raft_consensus.cc:504] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:44.969313 21728 leader_election.cc:288] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393), c933b2c37d69449cba36469dba40606b (127.6.72.66:46339)
Sep 25 00:45:44 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.66@KRBTEST.COM
Sep 25 00:45:44 dist-test-slave-w8nh krb5kdc[20145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.67@KRBTEST.COM
I20260925 00:45:44.984431 21730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:44.969873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:60909 (local address 127.6.72.66:46339)
0925 00:45:44.970279 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:44.970301 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:44.970346 (+ 45us) server_negotiation.cc:407] Connection header received
0925 00:45:44.971042 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:44.971048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:44.971382 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:44.971693 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:44.973075 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:44.974149 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:44.975419 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:44.975897 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:44.976100 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:44.979034 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:44.979041 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:44.979054 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:44.979090 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:44.981518 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:44.982388 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:44.982392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:44.982396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:44.982521 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:44.982843 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:44.982848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:44.982850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:44.983371 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:44.983561 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:44.983912 (+ 351us) server_negotiation.cc:299] Negotiation successful
0925 00:45:44.984197 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":142,"threads_started":1}
I20260925 00:45:44.984416 21725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:44.969611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:46339 (local address 127.6.72.65:60909)
0925 00:45:44.969874 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:44.969890 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:44.969999 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:44.971746 (+ 1747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:44.971752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:44.972274 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:44.972878 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:44.972902 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:44.974324 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:44.974327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:44.975253 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:44.975263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:44.976032 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:44.976036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:44.976302 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:44.977051 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:44.977062 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:44.978724 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:44.981637 (+ 2913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:44.981640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:44.981642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:44.981985 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:44.982635 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:44.982640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:44.982642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:44.982746 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:44.983478 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:44.983481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:44.983591 (+ 110us) client_negotiation.cc:769] Sending connection context
0925 00:45:44.983782 (+ 191us) client_negotiation.cc:240] Negotiation successful
0925 00:45:44.984209 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":208,"spinlock_wait_cycles":8704}
I20260925 00:45:44.984539 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:44.969772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:51191 (local address 127.6.72.67:33393)
0925 00:45:44.970126 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0925 00:45:44.970155 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0925 00:45:44.970516 (+ 361us) server_negotiation.cc:407] Connection header received
0925 00:45:44.971309 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:45:44.971315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:45:44.971689 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:45:44.971997 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:45:44.973097 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:44.974099 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:44.975401 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:45:44.975916 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:45:44.976109 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:44.979270 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:45:44.979275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:45:44.979286 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:45:44.979314 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:45:44.981351 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:44.982118 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:44.982123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:44.982127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:44.982253 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:45:44.982624 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:45:44.982627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:45:44.982628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:45:44.983058 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:45:44.983211 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 00:45:44.984015 (+ 804us) server_negotiation.cc:299] Negotiation successful
0925 00:45:44.984333 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":126,"threads_started":1}
I20260925 00:45:44.984560 21731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:44.969625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:33393 (local address 127.6.72.65:51191)
0925 00:45:44.970388 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:44.970401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:45:44.970537 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:45:44.972195 (+ 1658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:45:44.972198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:45:44.972449 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:45:44.972895 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:44.972907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:44.974261 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:44.974266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:44.975256 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:45:44.975266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:45:44.976058 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:45:44.976062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:45:44.976296 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:45:44.977006 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:45:44.977023 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:45:44.979082 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:45:44.981486 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:44.981493 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:44.981507 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:44.981985 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:44.982357 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:45:44.982359 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:45:44.982360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:45:44.982489 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:45:44.983182 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:45:44.983189 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:45:44.983475 (+ 286us) client_negotiation.cc:769] Sending connection context
0925 00:45:44.983873 (+ 398us) client_negotiation.cc:240] Negotiation successful
0925 00:45:44.984333 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":692,"spinlock_wait_cycles":8832,"thread_start_us":75,"threads_started":1}
W20260925 00:45:44.987622 21342 leader_election.cc:341] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393): Not found: Tablet not found: 6a854d1534d643fbbd0778b5333d4951
W20260925 00:45:44.987793 21341 leader_election.cc:341] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c933b2c37d69449cba36469dba40606b (127.6.72.66:46339): Not found: Tablet not found: 6a854d1534d643fbbd0778b5333d4951
I20260925 00:45:44.987914 21341 leader_election.cc:302] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53bea28e589e4e2e9516a0352c3f3029; no voters: 653e16553e2a47908c42b7985d5a3f65, c933b2c37d69449cba36469dba40606b
I20260925 00:45:44.988215 21728 raft_consensus.cc:2726] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 00:45:45.608466 21732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:45.607975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:43423)
0925 00:45:45.608308 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:45.608367 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":79,"threads_started":1}
I20260925 00:45:45.731773 21733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:45.731324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:43441)
0925 00:45:45.731616 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:45.731680 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":74,"threads_started":1}
I20260925 00:45:45.929723 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:45.929288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:54837)
0925 00:45:45.929545 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:45.929611 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
I20260925 00:45:46.607702 21735 raft_consensus.cc:482] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:45:46.607843 21735 raft_consensus.cc:504] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:46.608359 21735 leader_election.cc:288] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393), c933b2c37d69449cba36469dba40606b (127.6.72.66:46339)
W20260925 00:45:46.609162 21341 leader_election.cc:341] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c933b2c37d69449cba36469dba40606b (127.6.72.66:46339): Not found: Tablet not found: 6a854d1534d643fbbd0778b5333d4951
W20260925 00:45:46.609269 21342 leader_election.cc:341] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393): Not found: Tablet not found: 6a854d1534d643fbbd0778b5333d4951
I20260925 00:45:46.609416 21342 leader_election.cc:302] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53bea28e589e4e2e9516a0352c3f3029; no voters: 653e16553e2a47908c42b7985d5a3f65, c933b2c37d69449cba36469dba40606b
I20260925 00:45:46.609599 21735 raft_consensus.cc:2726] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 00:45:46.610111 21736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:46.609805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:48757)
0925 00:45:46.609986 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:46.610036 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":54,"threads_started":1}
I20260925 00:45:46.732983 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:46.732572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:54369)
0925 00:45:46.732816 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:46.732874 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260925 00:45:46.931030 21738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:46.930603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:38703)
0925 00:45:46.930866 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:46.930928 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":77,"threads_started":1}
I20260925 00:45:47.611303 21739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:47.610857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:49761)
0925 00:45:47.611128 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:47.611200 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260925 00:45:47.734143 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:47.733745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:50207)
0925 00:45:47.733985 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:47.734046 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260925 00:45:47.932510 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:47.931955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:39957)
0925 00:45:47.932338 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:47.932407 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":78,"threads_started":1}
I20260925 00:45:48.612855 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:48.612377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:36179)
0925 00:45:48.612698 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:48.612759 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":80,"threads_started":1}
I20260925 00:45:48.735558 21743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:48.735131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:37555)
0925 00:45:48.735398 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:48.735461 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":86,"threads_started":1}
I20260925 00:45:48.933956 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:48.933490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:42317)
0925 00:45:48.933777 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:48.933845 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":81,"threads_started":1}
I20260925 00:45:49.526044 21745 raft_consensus.cc:482] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:45:49.526185 21745 raft_consensus.cc:504] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:49.526558 21745 leader_election.cc:288] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393), c933b2c37d69449cba36469dba40606b (127.6.72.66:46339)
W20260925 00:45:49.527384 21341 leader_election.cc:341] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c933b2c37d69449cba36469dba40606b (127.6.72.66:46339): Not found: Tablet not found: 6a854d1534d643fbbd0778b5333d4951
W20260925 00:45:49.527527 21342 leader_election.cc:341] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393): Not found: Tablet not found: 6a854d1534d643fbbd0778b5333d4951
I20260925 00:45:49.527623 21342 leader_election.cc:302] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53bea28e589e4e2e9516a0352c3f3029; no voters: 653e16553e2a47908c42b7985d5a3f65, c933b2c37d69449cba36469dba40606b
I20260925 00:45:49.527801 21745 raft_consensus.cc:2726] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 00:45:49.614171 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:49.613712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:34123)
0925 00:45:49.614001 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:49.614060 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":90,"threads_started":1}
I20260925 00:45:49.737059 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:49.736521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:50461)
0925 00:45:49.736871 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:49.736959 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":108,"threads_started":1}
I20260925 00:45:49.935226 21748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:49.934801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:42749)
0925 00:45:49.935057 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:49.935118 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260925 00:45:50.615442 21749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:50.614993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:58003)
0925 00:45:50.615266 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:50.615331 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
I20260925 00:45:50.738329 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:50.737846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:47223)
0925 00:45:50.738158 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:50.738217 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":73,"threads_started":1}
I20260925 00:45:50.936530 21751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:50.935994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:37031)
0925 00:45:50.936370 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:50.936433 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":79,"threads_started":1}
I20260925 00:45:51.616842 21752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:51.616368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:33393)
0925 00:45:51.616673 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:51.616738 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":78,"threads_started":1}
I20260925 00:45:51.739571 21753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:51.739136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:34515)
0925 00:45:51.739395 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:51.739465 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
I20260925 00:45:51.937717 21754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:51.937294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:33055)
0925 00:45:51.937552 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:51.937621 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
I20260925 00:45:52.618033 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:52.617606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:47883)
0925 00:45:52.617875 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:52.617932 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
I20260925 00:45:52.740849 21756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:52.740429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:49611)
0925 00:45:52.740689 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:52.740750 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":78,"threads_started":1}
I20260925 00:45:52.939107 21757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:52.938540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:33495)
0925 00:45:52.938862 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:52.938974 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":117,"threads_started":1}
I20260925 00:45:53.620911 21758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:53.619495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:49047)
0925 00:45:53.620641 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:53.620754 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1036,"thread_start_us":143,"threads_started":1}
I20260925 00:45:53.634052 21759 raft_consensus.cc:482] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:45:53.634210 21759 raft_consensus.cc:504] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653e16553e2a47908c42b7985d5a3f65" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 33393 } } peers { permanent_uuid: "c933b2c37d69449cba36469dba40606b" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46339 } } peers { permanent_uuid: "53bea28e589e4e2e9516a0352c3f3029" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39877 } }
I20260925 00:45:53.634735 21759 leader_election.cc:288] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393), c933b2c37d69449cba36469dba40606b (127.6.72.66:46339)
W20260925 00:45:53.635517 21342 leader_election.cc:341] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 653e16553e2a47908c42b7985d5a3f65 (127.6.72.67:33393): Not found: Tablet not found: 6a854d1534d643fbbd0778b5333d4951
W20260925 00:45:53.635670 21341 leader_election.cc:341] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c933b2c37d69449cba36469dba40606b (127.6.72.66:46339): Not found: Tablet not found: 6a854d1534d643fbbd0778b5333d4951
I20260925 00:45:53.635779 21341 leader_election.cc:302] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53bea28e589e4e2e9516a0352c3f3029; no voters: 653e16553e2a47908c42b7985d5a3f65, c933b2c37d69449cba36469dba40606b
I20260925 00:45:53.635970 21759 raft_consensus.cc:2726] T 6a854d1534d643fbbd0778b5333d4951 P 53bea28e589e4e2e9516a0352c3f3029 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 00:45:53.742137 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:53.741682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:42453)
0925 00:45:53.741969 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:53.742032 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":89,"threads_started":1}
I20260925 00:45:53.940589 21761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:53.939981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:55671)
0925 00:45:53.940398 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:53.940486 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":102,"threads_started":1}
I20260925 00:45:54.622493 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:54.621873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:35489)
0925 00:45:54.622237 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:54.622330 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":114,"threads_started":1}
I20260925 00:45:54.743422 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:54.742963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:55321)
0925 00:45:54.743230 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:54.743288 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":79,"threads_started":1}
I20260925 00:45:54.941825 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:54.941399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:37765)
0925 00:45:54.941669 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:54.941728 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
I20260925 00:45:55.623898 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:55.623262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:44717)
0925 00:45:55.623657 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:55.623741 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":114,"threads_started":1}
I20260925 00:45:55.744716 21766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:55.744190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:60279)
0925 00:45:55.744560 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:55.744618 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":151,"threads_started":1}
I20260925 00:45:55.942924 21767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:55.942514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:55341)
0925 00:45:55.942777 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:55.942826 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":70,"threads_started":1}
I20260925 00:45:56.625308 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:56.624811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:51745)
0925 00:45:56.625115 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:56.625204 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":77,"threads_started":1}
I20260925 00:45:56.745863 21769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:56.745450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:52695)
0925 00:45:56.745712 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:56.745768 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":86,"threads_started":1}
I20260925 00:45:56.944118 21770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:56.943693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:55259)
0925 00:45:56.943956 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:56.944022 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":83,"threads_started":1}
I20260925 00:45:57.626622 21771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:57.626196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:34859)
0925 00:45:57.626462 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:57.626524 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":80,"threads_started":1}
I20260925 00:45:57.747062 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:57.746679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:43413)
0925 00:45:57.746915 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:57.746968 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":83,"threads_started":1}
I20260925 00:45:57.945216 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:57.944844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.67:60601)
0925 00:45:57.945070 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:57.945121 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260925 00:45:58.628191 21774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:58.627730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.65:58083)
0925 00:45:58.628014 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:58.628083 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":87,"threads_started":1}
I20260925 00:45:58.748456 21775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:45:58.747934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41357 (local address 127.6.72.66:58143)
0925 00:45:58.748204 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 00:45:58.748341 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":91,"threads_started":1}
I20260925 00:45:58.930384 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 21291
I20260925 00:45:58.936388 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 21458
I20260925 00:45:58.941926 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 21592
2026-09-25T00:45:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19508 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:45:59 dist-test-slave-w8nh krb5kdc[21778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:45:59 dist-test-slave-w8nh krb5kdc[21778](info): set up 2 sockets
Sep 25 00:45:59 dist-test-slave-w8nh krb5kdc[21778](info): commencing operation
krb5kdc: starting...
W20260925 00:46:00.040434 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:46:00 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:46:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:46:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:46:00.172171 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44583
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:44583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:46:00.290437 21791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:00.290757 21791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:00.290802 21791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:00.295136 21791 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 00:46:00.295217 21791 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 00:46:00.295234 21791 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 00:46:00.295248 21791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:46:00.295267 21791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:00.295286 21791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:00.295302 21791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:46:00.295378 21791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:46:00.301192 21791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:44583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:00.302443 21791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:00.303551 21791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:00.313299 21796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:00.313366 21791 server_base.cc:1034] running on GCE node
W20260925 00:46:00.313304 21797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:00.313304 21799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:00.314292 21791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:00.315557 21791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:00.316763 21791 hybrid_clock.cc:648] HybridClock initialized: now 1790297160316737 us; error 36 us; skew 500 ppm
Sep 25 00:46:00 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297160, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:00.319689 21791 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:46:00.320699 21791 webserver.cc:460] Webserver started at http://127.6.72.126:37023/ using document root <none> and password file <none>
I20260925 00:46:00.321291 21791 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:00.321382 21791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:00.321620 21791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:46:00.323711 21791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "9fec82d8b0514734a1f04f0c624887f8"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "3cb753e71ea13b5fa5a8c7b65c7fb7bd"
server_key_iv: "b0b0b94f03e36204b881a7d5a09220d5"
server_key_version: "encryptionkey@0"
I20260925 00:46:00.324357 21791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "9fec82d8b0514734a1f04f0c624887f8"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "3cb753e71ea13b5fa5a8c7b65c7fb7bd"
server_key_iv: "b0b0b94f03e36204b881a7d5a09220d5"
server_key_version: "encryptionkey@0"
I20260925 00:46:00.328131 21791 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:46:00.330636 21806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:00.331856 21791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:46:00.331990 21791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "9fec82d8b0514734a1f04f0c624887f8"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "3cb753e71ea13b5fa5a8c7b65c7fb7bd"
server_key_iv: "b0b0b94f03e36204b881a7d5a09220d5"
server_key_version: "encryptionkey@0"
I20260925 00:46:00.332108 21791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:00.346122 21791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:00.348922 21791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:00.349144 21791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:00.356725 21791 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:44583
I20260925 00:46:00.357120 21858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:44583 every 8 connection(s)
I20260925 00:46:00.357812 21791 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:46:00.359648 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 21791
I20260925 00:46:00.359802 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:46:00.360106 6433 external_mini_cluster.cc:1428] Setting key 169d79cd348b11758f82ed9c76559d97
I20260925 00:46:00.361465 21859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:46:00.368410 21859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Bootstrap starting.
Sep 25 00:46:00 dist-test-slave-w8nh krb5kdc[21778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:46:00.371351 21859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Neither blocks nor log segments found. Creating new log.
I20260925 00:46:00.372303 21859 log.cc:824] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Log is configured to *not* fsync() on all Append() calls
I20260925 00:46:00.374676 21859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: No bootstrap required, opened a new log
I20260925 00:46:00.375293 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:00.361666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36070 (local address 127.6.72.126:44583)
0925 00:46:00.362109 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:00.362132 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:00.362365 (+ 233us) server_negotiation.cc:407] Connection header received
0925 00:46:00.363167 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:00.363183 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:00.363556 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:00.363917 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:00.364791 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:00.365843 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:00.366745 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:00.367245 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:00.367438 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:00.370231 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:00.370236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:00.370251 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:00.370284 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:00.372183 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:00.372788 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:00.372801 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:00.372808 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:00.372923 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:00.373221 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:00.373225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:00.373227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:00.373650 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:00.373862 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:00.374182 (+ 320us) server_negotiation.cc:299] Negotiation successful
0925 00:46:00.374479 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":137,"threads_started":1}
I20260925 00:46:00.379172 21859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:00.379477 21859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:46:00.379585 21859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fec82d8b0514734a1f04f0c624887f8, State: Initialized, Role: FOLLOWER
I20260925 00:46:00.380077 21859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:00.380296 21859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:46:00.380400 21859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:46:00.380563 21859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:46:00.381680 21859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:00.382083 21859 leader_election.cc:302] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fec82d8b0514734a1f04f0c624887f8; no voters:
I20260925 00:46:00.382480 21859 leader_election.cc:288] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:46:00.382692 21864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:46:00.382961 21864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 1 LEADER]: Becoming Leader. State: Replica: 9fec82d8b0514734a1f04f0c624887f8, State: Running, Role: LEADER
I20260925 00:46:00.383327 21864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:00.384063 21859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:46:00.385676 21865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fec82d8b0514734a1f04f0c624887f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } } }
I20260925 00:46:00.385811 21865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: This master's current role is: LEADER
I20260925 00:46:00.386049 21866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fec82d8b0514734a1f04f0c624887f8. Latest consensus state: current_term: 1 leader_uuid: "9fec82d8b0514734a1f04f0c624887f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } } }
I20260925 00:46:00.386181 21866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: This master's current role is: LEADER
I20260925 00:46:00.386345 21873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:46:00.389798 21873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:46:00.395866 21873 catalog_manager.cc:1347] Generated new cluster ID: 11538457c3c24bc7b36aefb3e27a3991
I20260925 00:46:00.395961 21873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:46:00.414737 21873 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:46:00.415809 21873 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:46:00.435096 21873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Generated new TSK 0
I20260925 00:46:00.436010 21873 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:46:00.471048 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44583
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:46:00.586158 21885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:00.586511 21885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:00.586588 21885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:00.591014 21885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:00.591131 21885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:00.591234 21885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:46:00.597035 21885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:00.598270 21885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:00.599407 21885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:00.609935 21891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:00.609961 21890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:00.609941 21893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:00.610255 21885 server_base.cc:1034] running on GCE node
I20260925 00:46:00.610806 21885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:00.611555 21885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:00.612751 21885 hybrid_clock.cc:648] HybridClock initialized: now 1790297160612725 us; error 42 us; skew 500 ppm
Sep 25 00:46:00 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297160, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:00.615667 21885 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:46:00.616598 21885 webserver.cc:460] Webserver started at http://127.6.72.65:42835/ using document root <none> and password file <none>
I20260925 00:46:00.617131 21885 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:00.617187 21885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:00.617344 21885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:46:00.619423 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "ac4e51acafe84433a401f3df893bda8f"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "c9ccb94c928e61560a53306f879aab5d"
server_key_iv: "64c075a056374f31ee5cea851410b1ea"
server_key_version: "encryptionkey@0"
I20260925 00:46:00.619935 21885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "ac4e51acafe84433a401f3df893bda8f"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "c9ccb94c928e61560a53306f879aab5d"
server_key_iv: "64c075a056374f31ee5cea851410b1ea"
server_key_version: "encryptionkey@0"
I20260925 00:46:00.623616 21885 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 00:46:00.625972 21900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:00.627184 21885 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:46:00.627315 21885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "ac4e51acafe84433a401f3df893bda8f"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "c9ccb94c928e61560a53306f879aab5d"
server_key_iv: "64c075a056374f31ee5cea851410b1ea"
server_key_version: "encryptionkey@0"
I20260925 00:46:00.627424 21885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:00.639858 21885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:00.643146 21885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:00.643337 21885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:00.644045 21885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:46:00.645123 21885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:46:00.645184 21885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:00.645241 21885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:46:00.645278 21885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:00.654743 21885 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:35201
I20260925 00:46:00.654770 22013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:35201 every 8 connection(s)
I20260925 00:46:00.655751 21885 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:46:00.658386 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 21885
I20260925 00:46:00.658568 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:46:00.658950 6433 external_mini_cluster.cc:1428] Setting key e3e69366b8a44b7c20791a45adb08177
Sep 25 00:46:00 dist-test-slave-w8nh krb5kdc[21778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297160, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:46:00.673219 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:00.657794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:53861 (local address 127.6.72.126:44583)
0925 00:46:00.657974 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:00.657979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:00.658961 (+ 982us) server_negotiation.cc:407] Connection header received
0925 00:46:00.660742 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:00.660745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:00.660790 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:00.660883 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:00.662652 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:00.663250 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:00.664981 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:00.665239 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:00.665396 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:00.668558 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:00.668563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:00.668565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:00.668599 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:00.670317 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:00.670936 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:00.670941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:00.670944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:00.671030 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:00.671492 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:00.671501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:00.671502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:00.671699 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:00.671856 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:00.672693 (+ 837us) server_negotiation.cc:299] Negotiation successful
0925 00:46:00.672906 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 00:46:00.673810 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:00.658218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:53861)
0925 00:46:00.658768 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:00.658836 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:00.660491 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:00.661119 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:00.661132 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:00.661693 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:46:00.662443 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:00.662467 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:00.663386 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:00.663388 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:00.664836 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:00.664852 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:00.665438 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:00.665442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:00.665634 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:00.666406 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:46:00.666431 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:46:00.668229 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:46:00.670487 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:00.670494 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:00.670518 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:00.670810 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:00.671133 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:00.671136 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:00.671138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:00.671341 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:00.671887 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:46:00.671897 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:46:00.672362 (+ 465us) client_negotiation.cc:769] Sending connection context
0925 00:46:00.672555 (+ 193us) client_negotiation.cc:240] Negotiation successful
0925 00:46:00.672859 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":153,"threads_started":1}
I20260925 00:46:00.675539 22014 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44583
I20260925 00:46:00.675819 22014 heartbeater.cc:464] Registering TS with master...
I20260925 00:46:00.676429 22014 heartbeater.cc:511] Master 127.6.72.126:44583 requested a full tablet report, sending...
I20260925 00:46:00.678123 21822 ts_manager.cc:194] Registered new tserver with Master: ac4e51acafe84433a401f3df893bda8f (127.6.72.65:35201)
I20260925 00:46:00.679935 21822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:53861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:46:00.691370 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44583
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:46:00.804666 22019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:00.804981 22019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:00.805029 22019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:00.809876 22019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:00.809991 22019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:00.810133 22019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:46:00.816329 22019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:00.817605 22019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:00.818675 22019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:00.828421 22027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:00.828626 22024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:00.828511 22019 server_base.cc:1034] running on GCE node
W20260925 00:46:00.828394 22025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:00.829075 22019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:00.829676 22019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:00.830833 22019 hybrid_clock.cc:648] HybridClock initialized: now 1790297160830801 us; error 42 us; skew 500 ppm
Sep 25 00:46:00 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297160, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:00.833696 22019 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:46:00.834439 22019 webserver.cc:460] Webserver started at http://127.6.72.66:33079/ using document root <none> and password file <none>
I20260925 00:46:00.834944 22019 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:00.835045 22019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:00.835292 22019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:46:00.837412 22019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "6cc60a698bd14fea84ed074626d01996"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "8fdb0b254905be185faa3c84c61cfb12"
server_key_iv: "76ace560419756b70618ec62e43c9c25"
server_key_version: "encryptionkey@0"
I20260925 00:46:00.837963 22019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "6cc60a698bd14fea84ed074626d01996"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "8fdb0b254905be185faa3c84c61cfb12"
server_key_iv: "76ace560419756b70618ec62e43c9c25"
server_key_version: "encryptionkey@0"
I20260925 00:46:00.841444 22019 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:46:00.843662 22034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:00.844924 22019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:46:00.845039 22019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "6cc60a698bd14fea84ed074626d01996"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "8fdb0b254905be185faa3c84c61cfb12"
server_key_iv: "76ace560419756b70618ec62e43c9c25"
server_key_version: "encryptionkey@0"
I20260925 00:46:00.845139 22019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:00.866456 22019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:00.869818 22019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:00.870047 22019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:00.870704 22019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:46:00.871582 22019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:46:00.871628 22019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:00.871694 22019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:46:00.871728 22019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:00.881049 22019 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:44913
I20260925 00:46:00.881067 22147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:44913 every 8 connection(s)
I20260925 00:46:00.882025 22019 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:46:00.888527 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 22019
I20260925 00:46:00.888676 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:46:00.888967 6433 external_mini_cluster.cc:1428] Setting key a5f1210f632f9432758016aeec36d138
Sep 25 00:46:00 dist-test-slave-w8nh krb5kdc[21778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297160, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:46:00.899151 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:00.883959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:49867 (local address 127.6.72.126:44583)
0925 00:46:00.884120 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:00.884124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:00.885209 (+ 1085us) server_negotiation.cc:407] Connection header received
0925 00:46:00.887013 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:00.887016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:00.887085 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:00.887212 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:00.888706 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:00.889347 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:00.890904 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:00.891139 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:00.891258 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:00.894357 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:00.894365 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:00.894371 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:00.894409 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:00.896469 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:00.897130 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:00.897135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:00.897138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:00.897252 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:00.897749 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:00.897754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:00.897757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:00.898035 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:00.898154 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:00.898803 (+ 649us) server_negotiation.cc:299] Negotiation successful
0925 00:46:00.898963 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:46:00.899999 22150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:00.884404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:49867)
0925 00:46:00.885034 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:00.885098 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:00.886563 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:00.887409 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:00.887418 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:00.887820 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:46:00.888520 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:00.888543 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:00.889483 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:00.889487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:00.890773 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:00.890784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:00.891303 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:00.891308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:00.891497 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:00.892219 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:46:00.892277 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:46:00.894133 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:46:00.896624 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:00.896630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:00.896655 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:00.896990 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:00.897395 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:00.897398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:00.897399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:00.897617 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:00.898167 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:46:00.898173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:46:00.898491 (+ 318us) client_negotiation.cc:769] Sending connection context
0925 00:46:00.898675 (+ 184us) client_negotiation.cc:240] Negotiation successful
0925 00:46:00.898989 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":157,"threads_started":1}
I20260925 00:46:00.901345 22148 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44583
I20260925 00:46:00.901587 22148 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:46:00.902227 22148 heartbeater.cc:511] Master 127.6.72.126:44583 requested a full tablet report, sending...
I20260925 00:46:00.903410 21822 ts_manager.cc:194] Registered new tserver with Master: 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913)
I20260925 00:46:00.904058 21822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:49867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:46:00.919751 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44583
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:46:01.037169 22153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:01.037467 22153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:01.037535 22153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:01.041662 22153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:01.041743 22153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:01.041833 22153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:46:01.047153 22153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:01.048424 22153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:01.049644 22153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:01.060351 22161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:01.060549 22153 server_base.cc:1034] running on GCE node
W20260925 00:46:01.060353 22159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:01.060353 22158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:01.061065 22153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:01.061823 22153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:01.063247 22153 hybrid_clock.cc:648] HybridClock initialized: now 1790297161063213 us; error 68 us; skew 500 ppm
Sep 25 00:46:01 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297161, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:01.066452 22153 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:46:01.067421 22153 webserver.cc:460] Webserver started at http://127.6.72.67:35777/ using document root <none> and password file <none>
I20260925 00:46:01.067996 22153 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:01.068064 22153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:01.068315 22153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:46:01.070528 22153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "9bbe5b35aa2b4768b1713ed802bafe21"
format_stamp: "Formatted at 2026-09-25 00:46:01 on dist-test-slave-w8nh"
server_key: "8aca937fae95b889945c4ff2b73faaef"
server_key_iv: "75f7f58b0b524dd9d8c89b36e671255e"
server_key_version: "encryptionkey@0"
I20260925 00:46:01.071103 22153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "9bbe5b35aa2b4768b1713ed802bafe21"
format_stamp: "Formatted at 2026-09-25 00:46:01 on dist-test-slave-w8nh"
server_key: "8aca937fae95b889945c4ff2b73faaef"
server_key_iv: "75f7f58b0b524dd9d8c89b36e671255e"
server_key_version: "encryptionkey@0"
I20260925 00:46:01.075295 22153 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:46:01.078289 22168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:01.079764 22153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:46:01.079941 22153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "9bbe5b35aa2b4768b1713ed802bafe21"
format_stamp: "Formatted at 2026-09-25 00:46:01 on dist-test-slave-w8nh"
server_key: "8aca937fae95b889945c4ff2b73faaef"
server_key_iv: "75f7f58b0b524dd9d8c89b36e671255e"
server_key_version: "encryptionkey@0"
I20260925 00:46:01.080090 22153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:01.104852 22153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:01.108723 22153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:01.108958 22153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:01.109632 22153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:46:01.110783 22153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:46:01.110878 22153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:01.110968 22153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:46:01.111018 22153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:01.121253 22153 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:38415
I20260925 00:46:01.121268 22281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:38415 every 8 connection(s)
I20260925 00:46:01.122255 22153 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:46:01.127295 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 22153
I20260925 00:46:01.127460 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:46:01.127857 6433 external_mini_cluster.cc:1428] Setting key a0e0b95584bf92a3be7665d89d1580c5
Sep 25 00:46:01 dist-test-slave-w8nh krb5kdc[21778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297161, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:46:01.138720 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.124169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:58287 (local address 127.6.72.126:44583)
0925 00:46:01.124363 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:01.124367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:01.125835 (+ 1468us) server_negotiation.cc:407] Connection header received
0925 00:46:01.127077 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:01.127080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:01.127176 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:01.127333 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:01.129379 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.130290 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.132081 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.132444 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.132588 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.135511 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:01.135516 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:01.135518 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:01.135551 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:01.136949 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:01.137424 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:01.137426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:01.137428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:01.137498 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:01.137781 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:01.137783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:01.137784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:01.137969 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:01.138043 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:01.138500 (+ 457us) server_negotiation.cc:299] Negotiation successful
0925 00:46:01.138600 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:46:01.139571 22284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.124715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:58287)
0925 00:46:01.125688 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:01.125732 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:01.126799 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:01.127632 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:01.127644 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:01.128284 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:46:01.129206 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.129225 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.130460 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.130463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.131930 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.131945 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.132626 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.132631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.132829 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.133653 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:46:01.133681 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:46:01.135320 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:46:01.137061 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:01.137066 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:01.137079 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:01.137336 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:01.137575 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:01.137577 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:01.137578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:01.137705 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:01.138062 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:46:01.138066 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:46:01.138299 (+ 233us) client_negotiation.cc:769] Sending connection context
0925 00:46:01.138455 (+ 156us) client_negotiation.cc:240] Negotiation successful
0925 00:46:01.138710 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":572,"threads_started":1}
I20260925 00:46:01.140859 22282 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44583
I20260925 00:46:01.141142 22282 heartbeater.cc:464] Registering TS with master...
I20260925 00:46:01.141830 22282 heartbeater.cc:511] Master 127.6.72.126:44583 requested a full tablet report, sending...
I20260925 00:46:01.142895 21822 ts_manager.cc:194] Registered new tserver with Master: 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
I20260925 00:46:01.143443 21822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:58287
I20260925 00:46:01.154791 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:46:01.165781 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.157220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36080 (local address 127.6.72.126:44583)
0925 00:46:01.157363 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:01.157367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:01.157567 (+ 200us) server_negotiation.cc:407] Connection header received
0925 00:46:01.157660 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:01.157662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:01.157726 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:01.157812 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:01.158626 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.159159 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.160111 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.160493 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.160599 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.162119 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:01.162122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:01.162125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:01.162153 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:01.163829 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:01.164479 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:01.164483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:01.164485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:01.164600 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:01.164909 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:01.164915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:01.164919 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:01.165173 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:01.165280 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:01.165545 (+ 265us) server_negotiation.cc:299] Negotiation successful
0925 00:46:01.165652 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:46:01.169176 21822 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:46:01.172387 21822 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:46:01.188800 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.182176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52902 (local address 127.6.72.65:35201)
0925 00:46:01.182591 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:01.182599 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:01.182624 (+ 25us) server_negotiation.cc:407] Connection header received
0925 00:46:01.183231 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:01.183241 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:01.183415 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:01.183541 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:46:01.183950 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.185494 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.187340 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.187918 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.188067 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.188276 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:01.188504 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 00:46:01.188625 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":112,"threads_started":1}
I20260925 00:46:01.188831 22297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.182431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:38415 (local address 127.0.0.1:42452)
0925 00:46:01.182902 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:01.182915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:01.182994 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:01.183711 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:01.183713 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:01.183725 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:46:01.183839 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.183845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.185407 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.185412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.186926 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.186938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.188107 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.188111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.188278 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.188348 (+ 70us) client_negotiation.cc:769] Sending connection context
0925 00:46:01.188428 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 00:46:01.188615 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":87,"thread_start_us":104,"threads_started":1}
I20260925 00:46:01.188794 22294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.181840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35201 (local address 127.0.0.1:52902)
0925 00:46:01.182532 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:01.182546 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:01.182638 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:01.183648 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:01.183651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:01.183667 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:46:01.183815 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.183825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.185612 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.185614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.187178 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.187190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.188062 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.188066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.188254 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.188330 (+ 76us) client_negotiation.cc:769] Sending connection context
0925 00:46:01.188469 (+ 139us) client_negotiation.cc:240] Negotiation successful
0925 00:46:01.188642 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":344,"thread_start_us":99,"threads_started":1}
I20260925 00:46:01.189429 22298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.182867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42452 (local address 127.6.72.67:38415)
0925 00:46:01.183248 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:01.183261 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:01.183284 (+ 23us) server_negotiation.cc:407] Connection header received
0925 00:46:01.183362 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:01.183368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:01.183554 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:01.183741 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:46:01.183970 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.185159 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.187063 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.187936 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.188918 (+ 982us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.189086 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:01.189148 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 00:46:01.189265 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":156,"threads_started":1}
I20260925 00:46:01.189767 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.181673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:44913 (local address 127.0.0.1:50208)
0925 00:46:01.182356 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:01.182412 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:01.182547 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:01.183416 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:01.183422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:01.183445 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:46:01.183590 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.183599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.185593 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.185597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.187178 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.187190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.188586 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.188591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.188677 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.188702 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 00:46:01.189528 (+ 826us) client_negotiation.cc:240] Negotiation successful
0925 00:46:01.189625 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":180,"thread_start_us":97,"threads_started":1}
I20260925 00:46:01.192526 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.181807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50208 (local address 127.6.72.66:44913)
0925 00:46:01.182833 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:01.182845 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:01.182866 (+ 21us) server_negotiation.cc:407] Connection header received
0925 00:46:01.182945 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:01.182952 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:01.183185 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:01.183431 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:46:01.183731 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.185413 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.187339 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.188202 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.188343 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.188519 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:01.190046 (+ 1527us) server_negotiation.cc:299] Negotiation successful
0925 00:46:01.192344 (+ 2298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":926,"thread_start_us":156,"threads_started":1}
I20260925 00:46:01.192210 22216 tablet_service.cc:1467] Processing CreateTablet for tablet 67989a68d6f24870bdbbada1ee1e3739 (DEFAULT_TABLE table=test-table [id=0c5928f8a81d43bbb03865a68479d706]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:46:01.192214 21948 tablet_service.cc:1467] Processing CreateTablet for tablet 67989a68d6f24870bdbbada1ee1e3739 (DEFAULT_TABLE table=test-table [id=0c5928f8a81d43bbb03865a68479d706]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:46:01.193550 22216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67989a68d6f24870bdbbada1ee1e3739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:46:01.194222 21948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67989a68d6f24870bdbbada1ee1e3739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:46:01.195737 22082 tablet_service.cc:1467] Processing CreateTablet for tablet 67989a68d6f24870bdbbada1ee1e3739 (DEFAULT_TABLE table=test-table [id=0c5928f8a81d43bbb03865a68479d706]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:46:01.196971 22082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67989a68d6f24870bdbbada1ee1e3739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:46:01.200462 22299 tablet_bootstrap.cc:492] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: Bootstrap starting.
I20260925 00:46:01.202772 22300 tablet_bootstrap.cc:492] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Bootstrap starting.
I20260925 00:46:01.203327 22299 tablet_bootstrap.cc:654] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: Neither blocks nor log segments found. Creating new log.
I20260925 00:46:01.204818 22301 tablet_bootstrap.cc:492] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: Bootstrap starting.
I20260925 00:46:01.205317 22299 log.cc:824] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: Log is configured to *not* fsync() on all Append() calls
I20260925 00:46:01.205391 22300 tablet_bootstrap.cc:654] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Neither blocks nor log segments found. Creating new log.
I20260925 00:46:01.206218 22300 log.cc:824] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Log is configured to *not* fsync() on all Append() calls
I20260925 00:46:01.207793 22301 tablet_bootstrap.cc:654] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: Neither blocks nor log segments found. Creating new log.
I20260925 00:46:01.208026 22299 tablet_bootstrap.cc:492] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: No bootstrap required, opened a new log
I20260925 00:46:01.208303 22299 ts_tablet_manager.cc:1397] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 00:46:01.208770 22301 log.cc:824] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: Log is configured to *not* fsync() on all Append() calls
I20260925 00:46:01.209731 22300 tablet_bootstrap.cc:492] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: No bootstrap required, opened a new log
I20260925 00:46:01.209940 22300 ts_tablet_manager.cc:1397] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 00:46:01.210973 22301 tablet_bootstrap.cc:492] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: No bootstrap required, opened a new log
I20260925 00:46:01.211176 22301 ts_tablet_manager.cc:1397] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 00:46:01.212625 22299 raft_consensus.cc:348] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:01.212960 22299 raft_consensus.cc:374] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:46:01.213058 22299 raft_consensus.cc:729] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bbe5b35aa2b4768b1713ed802bafe21, State: Initialized, Role: FOLLOWER
I20260925 00:46:01.213600 22299 consensus_queue.cc:260] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:01.214190 22300 raft_consensus.cc:348] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:01.214401 22282 heartbeater.cc:503] Master 127.6.72.126:44583 was elected leader, sending a full tablet report...
I20260925 00:46:01.214454 22300 raft_consensus.cc:374] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:46:01.214533 22300 raft_consensus.cc:729] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac4e51acafe84433a401f3df893bda8f, State: Initialized, Role: FOLLOWER
I20260925 00:46:01.214950 22300 consensus_queue.cc:260] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:01.215561 22014 heartbeater.cc:503] Master 127.6.72.126:44583 was elected leader, sending a full tablet report...
I20260925 00:46:01.215579 22301 raft_consensus.cc:348] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:01.215843 22301 raft_consensus.cc:374] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:46:01.215919 22301 raft_consensus.cc:729] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc60a698bd14fea84ed074626d01996, State: Initialized, Role: FOLLOWER
I20260925 00:46:01.215984 22300 ts_tablet_manager.cc:1428] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 00:46:01.216337 22301 consensus_queue.cc:260] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:01.217113 22148 heartbeater.cc:503] Master 127.6.72.126:44583 was elected leader, sending a full tablet report...
I20260925 00:46:01.217353 22301 ts_tablet_manager.cc:1428] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 00:46:01.217864 22299 ts_tablet_manager.cc:1428] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260925 00:46:01.373492 22283 tablet.cc:2369] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:46:01.383239 22149 tablet.cc:2369] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:46:01.407416 22015 tablet.cc:2369] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:46:01.460842 22306 raft_consensus.cc:482] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:46:01.461120 22306 raft_consensus.cc:504] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:01.462344 22306 leader_election.cc:288] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913), 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
I20260925 00:46:01.467051 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.462838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:45717 (local address 127.6.72.66:44913)
0925 00:46:01.463032 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:01.463036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:01.463376 (+ 340us) server_negotiation.cc:407] Connection header received
0925 00:46:01.463637 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:01.463641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:01.463706 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:01.463822 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:46:01.464126 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.464817 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.466057 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.466462 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.466590 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.466658 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:01.466830 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 00:46:01.466910 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 00:46:01.467087 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.462763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:44913 (local address 127.6.72.65:45717)
0925 00:46:01.463243 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:01.463263 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:01.463424 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:01.463833 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:01.463837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:01.463854 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:46:01.464001 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.464007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.464959 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.464965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.465919 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.465929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.466590 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.466594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.466692 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.466715 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 00:46:01.466814 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 00:46:01.466910 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":22,"thread_start_us":268,"threads_started":1}
I20260925 00:46:01.467612 22298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.463601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:59231 (local address 127.6.72.67:38415)
0925 00:46:01.463804 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:01.463813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:01.463885 (+ 72us) server_negotiation.cc:407] Connection header received
0925 00:46:01.464069 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:01.464073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:01.464152 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:01.464249 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:46:01.464612 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.465247 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.466434 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.466945 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.467083 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.467155 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:01.467391 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 00:46:01.467474 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260925 00:46:01.467644 22309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.463402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:38415 (local address 127.6.72.65:59231)
0925 00:46:01.463799 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:01.463815 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:01.463929 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:01.464300 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:01.464304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:01.464324 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:46:01.464482 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.464489 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.465385 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.465389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.466267 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:01.466277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.467083 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.467088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:01.467204 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.467235 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 00:46:01.467381 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 00:46:01.467465 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":1,"thread_start_us":134,"threads_started":1}
I20260925 00:46:01.467860 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67989a68d6f24870bdbbada1ee1e3739" candidate_uuid: "ac4e51acafe84433a401f3df893bda8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc60a698bd14fea84ed074626d01996" is_pre_election: true
I20260925 00:46:01.468315 22102 raft_consensus.cc:2454] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac4e51acafe84433a401f3df893bda8f in term 0.
I20260925 00:46:01.468423 22236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67989a68d6f24870bdbbada1ee1e3739" candidate_uuid: "ac4e51acafe84433a401f3df893bda8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" is_pre_election: true
I20260925 00:46:01.468710 22236 raft_consensus.cc:2454] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac4e51acafe84433a401f3df893bda8f in term 0.
I20260925 00:46:01.468911 21904 leader_election.cc:302] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc60a698bd14fea84ed074626d01996, ac4e51acafe84433a401f3df893bda8f; no voters:
I20260925 00:46:01.469241 22306 raft_consensus.cc:2781] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:46:01.469319 22306 raft_consensus.cc:482] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:46:01.469349 22306 raft_consensus.cc:3037] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:46:01.470465 22306 raft_consensus.cc:504] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:01.470834 22306 leader_election.cc:288] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 1 election: Requested vote from peers 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913), 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
I20260925 00:46:01.471290 22102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67989a68d6f24870bdbbada1ee1e3739" candidate_uuid: "ac4e51acafe84433a401f3df893bda8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc60a698bd14fea84ed074626d01996"
I20260925 00:46:01.471422 22102 raft_consensus.cc:3037] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:46:01.471449 22236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67989a68d6f24870bdbbada1ee1e3739" candidate_uuid: "ac4e51acafe84433a401f3df893bda8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bbe5b35aa2b4768b1713ed802bafe21"
I20260925 00:46:01.471601 22236 raft_consensus.cc:3037] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:46:01.472661 22102 raft_consensus.cc:2454] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac4e51acafe84433a401f3df893bda8f in term 1.
I20260925 00:46:01.472752 22236 raft_consensus.cc:2454] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac4e51acafe84433a401f3df893bda8f in term 1.
I20260925 00:46:01.473038 21904 leader_election.cc:302] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc60a698bd14fea84ed074626d01996, ac4e51acafe84433a401f3df893bda8f; no voters:
I20260925 00:46:01.473241 22306 raft_consensus.cc:2781] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:46:01.473532 22306 raft_consensus.cc:686] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 1 LEADER]: Becoming Leader. State: Replica: ac4e51acafe84433a401f3df893bda8f, State: Running, Role: LEADER
I20260925 00:46:01.473930 22306 consensus_queue.cc:237] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:01.477636 21823 catalog_manager.cc:5495] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ac4e51acafe84433a401f3df893bda8f (127.6.72.65). New cstate: current_term: 1 leader_uuid: "ac4e51acafe84433a401f3df893bda8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } health_report { overall_health: HEALTHY } } }
I20260925 00:46:01.555220 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.550977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.6.72.65:35201)
0925 00:46:01.551140 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:01.551144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:01.551160 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:46:01.551251 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:01.551254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:01.551321 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:01.551438 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:46:01.551681 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.552232 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.553265 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:01.553497 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:01.553672 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:01.554056 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:46:01.554717 (+ 661us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:46:01.554816 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:01.554950 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 00:46:01.555029 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 00:46:01.562129 22236 raft_consensus.cc:1261] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 1 FOLLOWER]: Refusing update from remote peer ac4e51acafe84433a401f3df893bda8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:46:01.562130 22102 raft_consensus.cc:1261] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996 [term 1 FOLLOWER]: Refusing update from remote peer ac4e51acafe84433a401f3df893bda8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:46:01.563020 22306 consensus_queue.cc:1035] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:46:01.563288 22310 consensus_queue.cc:1035] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:46:01.572163 22315 mvcc.cc:205] Tried to move back new op lower bound from 7333057173749157888 to 7333057173400309760. Current Snapshot: MvccSnapshot[applied={T|T < 7333057173749157888}]
I20260925 00:46:01.581960 21823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:46:01.582201 21823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:46:01.585528 21823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Sending DeleteTablet for 3 replicas of tablet 67989a68d6f24870bdbbada1ee1e3739
I20260925 00:46:01.586685 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 21885
I20260925 00:46:01.587488 22082 tablet_service.cc:1514] Processing DeleteTablet for tablet 67989a68d6f24870bdbbada1ee1e3739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:46:01 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:50208
I20260925 00:46:01.588138 22323 tablet_replica.cc:330] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: stopping tablet replica
I20260925 00:46:01.588709 22323 raft_consensus.cc:2229] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:46:01.589100 22216 tablet_service.cc:1514] Processing DeleteTablet for tablet 67989a68d6f24870bdbbada1ee1e3739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:46:01 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:42452
I20260925 00:46:01.589205 22323 raft_consensus.cc:2258] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:46:01.591691 22324 tablet_replica.cc:330] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: stopping tablet replica
I20260925 00:46:01.592119 22324 raft_consensus.cc:2229] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:46:01.592566 22324 raft_consensus.cc:2258] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:46:01.592702 22323 ts_tablet_manager.cc:1905] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:46:01.594455 22324 ts_tablet_manager.cc:1905] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:46:01.596155 22323 ts_tablet_manager.cc:1918] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260925 00:46:01.596179 21808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:35201 (error 108)
I20260925 00:46:01.596375 22323 log.cc:1196] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/67989a68d6f24870bdbbada1ee1e3739
I20260925 00:46:01.596580 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 22019
I20260925 00:46:01.596814 22323 ts_tablet_manager.cc:1939] T 67989a68d6f24870bdbbada1ee1e3739 P 6cc60a698bd14fea84ed074626d01996: Deleting consensus metadata
I20260925 00:46:01.597563 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.597158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35201 (local address 127.0.0.1:52924)
0925 00:46:01.597348 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:01.597454 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:35201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260925 00:46:01.597759 21808 catalog_manager.cc:4584] TS ac4e51acafe84433a401f3df893bda8f (127.6.72.65:35201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 67989a68d6f24870bdbbada1ee1e3739: Network error: Client connection negotiation failed: client connection to 127.6.72.65:35201: connect: Connection refused (error 111)
I20260925 00:46:01.597859 22324 ts_tablet_manager.cc:1918] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:46:01.597949 22324 log.cc:1196] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/67989a68d6f24870bdbbada1ee1e3739
I20260925 00:46:01.598220 22324 ts_tablet_manager.cc:1939] T 67989a68d6f24870bdbbada1ee1e3739 P 9bbe5b35aa2b4768b1713ed802bafe21: Deleting consensus metadata
I20260925 00:46:01.599536 21810 catalog_manager.cc:4841] TS 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415): tablet 67989a68d6f24870bdbbada1ee1e3739 (table test-table [id=0c5928f8a81d43bbb03865a68479d706]) successfully deleted
I20260925 00:46:01.603335 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 22153
I20260925 00:46:01.604417 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:01.603759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:44913 (local address 127.0.0.1:50216)
0925 00:46:01.604081 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:01.604136 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.66:44913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228}
I20260925 00:46:01.609637 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 21791
I20260925 00:46:21.616468 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44583
--webserver_interface=127.6.72.126
--webserver_port=37023
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:44583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:46:21.735579 22326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:21.735863 22326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:21.735905 22326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:21.740049 22326 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 00:46:21.740118 22326 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 00:46:21.740155 22326 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 00:46:21.740170 22326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:46:21.740190 22326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:21.740207 22326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:21.740223 22326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:46:21.740298 22326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:46:21.745950 22326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:44583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=37023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:21.747010 22326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:21.747979 22326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 00:46:21.761353 22326 server_base.cc:1034] running on GCE node
W20260925 00:46:21.761324 22333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:21.761538 22335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:21.761310 22332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:21.762197 22326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:21.763443 22326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:21.764648 22326 hybrid_clock.cc:648] HybridClock initialized: now 1790297181764630 us; error 31 us; skew 500 ppm
Sep 25 00:46:21 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:21.767457 22326 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:46:21.768424 22326 webserver.cc:460] Webserver started at http://127.6.72.126:37023/ using document root <none> and password file <none>
I20260925 00:46:21.769025 22326 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:21.769132 22326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:21.773136 22326 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260925 00:46:21.775462 22342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:21.776724 22326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 00:46:21.776867 22326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "9fec82d8b0514734a1f04f0c624887f8"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "3cb753e71ea13b5fa5a8c7b65c7fb7bd"
server_key_iv: "b0b0b94f03e36204b881a7d5a09220d5"
server_key_version: "encryptionkey@0"
I20260925 00:46:21.777590 22326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:21.798796 22326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:21.801977 22326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:21.802199 22326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:21.809320 22326 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:44583
I20260925 00:46:21.809657 22394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:44583 every 8 connection(s)
I20260925 00:46:21.810462 22326 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:46:21.814311 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 22326
I20260925 00:46:21.814788 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:35201
--local_ip_for_outbound_sockets=127.6.72.65
--tserver_master_addrs=127.6.72.126:44583
--webserver_port=42835
--webserver_interface=127.6.72.65
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:46:21.815248 22395 sys_catalog.cc:263] Verifying existing consensus state
I20260925 00:46:21.817544 22395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Bootstrap starting.
I20260925 00:46:21.841998 22395 log.cc:824] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Log is configured to *not* fsync() on all Append() calls
I20260925 00:46:21.852960 22395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:46:21.853398 22395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Bootstrap complete.
I20260925 00:46:21.859145 22395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:21.859750 22395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fec82d8b0514734a1f04f0c624887f8, State: Initialized, Role: FOLLOWER
I20260925 00:46:21.860402 22395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:21.860570 22395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:46:21.860683 22395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:46:21.860805 22395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 1 FOLLOWER]: Advancing to term 2
I20260925 00:46:21.862850 22395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:21.863216 22395 leader_election.cc:302] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fec82d8b0514734a1f04f0c624887f8; no voters:
I20260925 00:46:21.863544 22395 leader_election.cc:288] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 00:46:21.863684 22399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 00:46:21.863943 22399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 2 LEADER]: Becoming Leader. State: Replica: 9fec82d8b0514734a1f04f0c624887f8, State: Running, Role: LEADER
I20260925 00:46:21.864355 22399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:21.864866 22395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:46:21.865895 22401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fec82d8b0514734a1f04f0c624887f8. Latest consensus state: current_term: 2 leader_uuid: "9fec82d8b0514734a1f04f0c624887f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } } }
I20260925 00:46:21.866034 22401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: This master's current role is: LEADER
I20260925 00:46:21.866292 22400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9fec82d8b0514734a1f04f0c624887f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } } }
I20260925 00:46:21.866369 22400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: This master's current role is: LEADER
I20260925 00:46:21.866396 22408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:46:21.871218 22408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:46:21.871951 22408 catalog_manager.cc:1259] Loaded cluster ID: 11538457c3c24bc7b36aefb3e27a3991
I20260925 00:46:21.872020 22408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:46:21.874670 22408 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:46:21.875329 22408 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Loaded TSK: 0
I20260925 00:46:21.895330 22408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Generated new TSK 1
I20260925 00:46:21.895637 22408 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 00:46:21.951887 22397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:21.952154 22397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:21.952199 22397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:21.956404 22397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:21.956470 22397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:21.956560 22397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:46:21.961856 22397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:35201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=42835
--tserver_master_addrs=127.6.72.126:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:21.962908 22397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:21.963958 22397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:21.974313 22425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:21.974335 22423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:21.974675 22397 server_base.cc:1034] running on GCE node
W20260925 00:46:21.974354 22422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:21.975210 22397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:21.975922 22397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:21.977111 22397 hybrid_clock.cc:648] HybridClock initialized: now 1790297181977073 us; error 58 us; skew 500 ppm
Sep 25 00:46:21 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:21.979888 22397 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:46:21.980845 22397 webserver.cc:460] Webserver started at http://127.6.72.65:42835/ using document root <none> and password file <none>
I20260925 00:46:21.981421 22397 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:21.981493 22397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:21.985827 22397 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:46:21.988174 22432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:21.989567 22397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:46:21.989717 22397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "ac4e51acafe84433a401f3df893bda8f"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "c9ccb94c928e61560a53306f879aab5d"
server_key_iv: "64c075a056374f31ee5cea851410b1ea"
server_key_version: "encryptionkey@0"
I20260925 00:46:21.990286 22397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:22.005342 22397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:22.008867 22397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:22.009094 22397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:22.009709 22397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:46:22.010941 22439 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 00:46:22.015019 22397 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 00:46:22.015081 22397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260925 00:46:22.015130 22397 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 00:46:22.016844 22397 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 00:46:22.016899 22397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260925 00:46:22.017158 22439 tablet_bootstrap.cc:492] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Bootstrap starting.
I20260925 00:46:22.027992 22397 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:35201
I20260925 00:46:22.028093 22546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:35201 every 8 connection(s)
I20260925 00:46:22.029281 22397 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:46:22.032680 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 22397
I20260925 00:46:22.033213 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:44913
--local_ip_for_outbound_sockets=127.6.72.66
--tserver_master_addrs=127.6.72.126:44583
--webserver_port=33079
--webserver_interface=127.6.72.66
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:46:22.040096 22439 log.cc:824] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Log is configured to *not* fsync() on all Append() calls
Sep 25 00:46:22 dist-test-slave-w8nh krb5kdc[21778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:46:22.047569 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.031356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:44843 (local address 127.6.72.126:44583)
0925 00:46:22.031733 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.031753 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.032089 (+ 336us) server_negotiation.cc:407] Connection header received
0925 00:46:22.033624 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.033640 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.034062 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.034454 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:22.036035 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.037510 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.039007 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.039489 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.039686 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.042450 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:22.042455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:22.042467 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:22.042508 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:22.044326 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:22.044874 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:22.044883 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:22.044892 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:22.045018 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:22.045377 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:22.045382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:22.045384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:22.045739 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:22.045885 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.046416 (+ 531us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.046689 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":146,"threads_started":1}
I20260925 00:46:22.047582 22550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.031384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:44843)
0925 00:46:22.031924 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:22.031965 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:22.033088 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:22.034557 (+ 1469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:22.034569 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:22.035115 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:46:22.035817 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.035841 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.037672 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.037675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.038843 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.038855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.039645 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.039648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.039832 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.040537 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:46:22.040554 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:46:22.042235 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:46:22.044453 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:22.044458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:22.044472 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:22.044742 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:22.045133 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:22.045138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:22.045140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:22.045280 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:22.045857 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:46:22.045862 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:46:22.046114 (+ 252us) client_negotiation.cc:769] Sending connection context
0925 00:46:22.046291 (+ 177us) client_negotiation.cc:240] Negotiation successful
0925 00:46:22.046643 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":140,"threads_started":1}
I20260925 00:46:22.049997 22439 tablet_bootstrap.cc:492] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:46:22.050148 22547 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44583
I20260925 00:46:22.050391 22547 heartbeater.cc:464] Registering TS with master...
I20260925 00:46:22.050419 22439 tablet_bootstrap.cc:492] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Bootstrap complete.
I20260925 00:46:22.050876 22547 heartbeater.cc:511] Master 127.6.72.126:44583 requested a full tablet report, sending...
I20260925 00:46:22.051827 22439 ts_tablet_manager.cc:1397] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Time spent bootstrapping tablet: real 0.035s user 0.029s sys 0.000s
I20260925 00:46:22.053298 22359 ts_manager.cc:194] Registered new tserver with Master: ac4e51acafe84433a401f3df893bda8f (127.6.72.65:35201)
I20260925 00:46:22.055788 22439 raft_consensus.cc:348] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.056494 22439 raft_consensus.cc:729] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac4e51acafe84433a401f3df893bda8f, State: Initialized, Role: FOLLOWER
I20260925 00:46:22.056927 22359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:44843
I20260925 00:46:22.057065 22439 consensus_queue.cc:260] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.058107 22439 ts_tablet_manager.cc:1428] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260925 00:46:22.059842 22547 heartbeater.cc:503] Master 127.6.72.126:44583 was elected leader, sending a full tablet report...
I20260925 00:46:22.063158 22554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.057043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47646 (local address 127.6.72.65:35201)
0925 00:46:22.057641 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.057653 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.058304 (+ 651us) server_negotiation.cc:407] Connection header received
0925 00:46:22.058377 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.058383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.058890 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.059057 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:46:22.059398 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.060477 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.062031 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.062536 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.062623 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.062722 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.062964 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.063051 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":114,"threads_started":1}
I20260925 00:46:22.063195 22553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.056230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35201 (local address 127.0.0.1:47646)
0925 00:46:22.057899 (+ 1669us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:22.057946 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:22.058097 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:22.059081 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:22.059087 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:22.059113 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:46:22.059258 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.059267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.060596 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.060600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.061903 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.061914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.062660 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.062664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.062780 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.062819 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 00:46:22.062932 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 00:46:22.063055 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1482,"thread_start_us":107,"threads_started":1}
I20260925 00:46:22.064347 22481 tablet_service.cc:1514] Processing DeleteTablet for tablet 67989a68d6f24870bdbbada1ee1e3739 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:46:01 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:47646
I20260925 00:46:22.065209 22556 tablet_replica.cc:330] stopping tablet replica
I20260925 00:46:22.065536 22556 raft_consensus.cc:2229] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:46:22.065788 22556 raft_consensus.cc:2258] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:46:22.066835 22556 ts_tablet_manager.cc:1905] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:46:22.070498 22556 ts_tablet_manager.cc:1918] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:46:22.070580 22556 log.cc:1196] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal/wals/67989a68d6f24870bdbbada1ee1e3739
I20260925 00:46:22.070847 22556 ts_tablet_manager.cc:1939] T 67989a68d6f24870bdbbada1ee1e3739 P ac4e51acafe84433a401f3df893bda8f: Deleting consensus metadata
I20260925 00:46:22.072866 22344 catalog_manager.cc:4841] TS ac4e51acafe84433a401f3df893bda8f (127.6.72.65:35201): tablet 67989a68d6f24870bdbbada1ee1e3739 (table test-table [id=0c5928f8a81d43bbb03865a68479d706]) successfully deleted
W20260925 00:46:22.153092 22551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:22.153405 22551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:22.153479 22551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:22.157769 22551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:22.157858 22551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:22.157953 22551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:46:22.163268 22551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:44913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=33079
--tserver_master_addrs=127.6.72.126:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:22.164567 22551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:22.165792 22551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:22.175740 22562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:22.175853 22551 server_base.cc:1034] running on GCE node
W20260925 00:46:22.175738 22561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:22.175734 22564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:22.176412 22551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:22.177103 22551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:22.178311 22551 hybrid_clock.cc:648] HybridClock initialized: now 1790297182178289 us; error 26 us; skew 500 ppm
Sep 25 00:46:22 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297182, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:22.181254 22551 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:46:22.182188 22551 webserver.cc:460] Webserver started at http://127.6.72.66:33079/ using document root <none> and password file <none>
I20260925 00:46:22.182803 22551 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:22.182881 22551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:22.186708 22551 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:46:22.188761 22571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:22.190004 22551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:46:22.190147 22551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "6cc60a698bd14fea84ed074626d01996"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "8fdb0b254905be185faa3c84c61cfb12"
server_key_iv: "76ace560419756b70618ec62e43c9c25"
server_key_version: "encryptionkey@0"
I20260925 00:46:22.190788 22551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:22.209079 22551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:22.212769 22551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:22.212987 22551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:22.213615 22551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:46:22.214476 22551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:46:22.214532 22551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:22.214603 22551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:46:22.214635 22551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:22.224045 22551 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:44913
I20260925 00:46:22.224078 22684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:44913 every 8 connection(s)
I20260925 00:46:22.225122 22551 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:46:22.231278 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 22551
I20260925 00:46:22.231767 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:38415
--local_ip_for_outbound_sockets=127.6.72.67
--tserver_master_addrs=127.6.72.126:44583
--webserver_port=35777
--webserver_interface=127.6.72.67
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 00:46:22 dist-test-slave-w8nh krb5kdc[21778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297182, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:46:22.240126 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.226949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:46129 (local address 127.6.72.126:44583)
0925 00:46:22.227103 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.227107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.227860 (+ 753us) server_negotiation.cc:407] Connection header received
0925 00:46:22.229235 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.229239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.229298 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.229379 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:22.230727 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.231480 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.232941 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.233256 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.233375 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.236408 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:22.236411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:22.236414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:22.236443 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:22.237994 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:22.238554 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:22.238561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:22.238564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:22.238640 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:22.238998 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:22.239001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:22.239003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:22.239188 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:22.239294 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.239808 (+ 514us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.239947 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:46:22.240995 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.227275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:46129)
0925 00:46:22.227701 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:22.227744 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:22.229022 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:22.229565 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:22.229573 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:22.229937 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:46:22.230404 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.230417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.231654 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.231657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.232796 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.232807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.233464 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.233470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.233699 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.234374 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:46:22.234401 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:46:22.236163 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:46:22.238135 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:22.238141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:22.238158 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:22.238434 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:22.238743 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:22.238746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:22.238748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:22.238892 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:22.239305 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:46:22.239313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:46:22.239566 (+ 253us) client_negotiation.cc:769] Sending connection context
0925 00:46:22.239753 (+ 187us) client_negotiation.cc:240] Negotiation successful
0925 00:46:22.240098 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":106,"threads_started":1}
I20260925 00:46:22.242249 22685 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44583
I20260925 00:46:22.242542 22685 heartbeater.cc:464] Registering TS with master...
I20260925 00:46:22.243134 22685 heartbeater.cc:511] Master 127.6.72.126:44583 requested a full tablet report, sending...
I20260925 00:46:22.244283 22359 ts_manager.cc:194] Registered new tserver with Master: 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913)
I20260925 00:46:22.245040 22359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:46129
W20260925 00:46:22.345747 22688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:22.346012 22688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:22.346055 22688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:22.350003 22688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:22.350063 22688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:22.350121 22688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:46:22.355443 22688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:38415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=35777
--tserver_master_addrs=127.6.72.126:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:22.356607 22688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:22.357690 22688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:22.368058 22696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:22.368100 22693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:22.368263 22688 server_base.cc:1034] running on GCE node
W20260925 00:46:22.368121 22694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:22.368757 22688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:22.369374 22688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:22.370592 22688 hybrid_clock.cc:648] HybridClock initialized: now 1790297182370554 us; error 48 us; skew 500 ppm
Sep 25 00:46:22 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297182, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:22.373533 22688 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:46:22.374456 22688 webserver.cc:460] Webserver started at http://127.6.72.67:35777/ using document root <none> and password file <none>
I20260925 00:46:22.375025 22688 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:22.375130 22688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:22.378980 22688 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:46:22.381177 22703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:22.382761 22688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:46:22.382920 22688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "9bbe5b35aa2b4768b1713ed802bafe21"
format_stamp: "Formatted at 2026-09-25 00:46:01 on dist-test-slave-w8nh"
server_key: "8aca937fae95b889945c4ff2b73faaef"
server_key_iv: "75f7f58b0b524dd9d8c89b36e671255e"
server_key_version: "encryptionkey@0"
I20260925 00:46:22.383550 22688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:22.397346 22688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:22.400658 22688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:22.400810 22688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:22.401330 22688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:46:22.402086 22688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:46:22.402130 22688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:22.402169 22688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:46:22.402190 22688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:22.411523 22688 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:38415
I20260925 00:46:22.411550 22816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:38415 every 8 connection(s)
I20260925 00:46:22.412616 22688 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:46:22.418955 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 22688
Sep 25 00:46:22 dist-test-slave-w8nh krb5kdc[21778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297182, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:46:22.427697 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.414508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:52733 (local address 127.6.72.126:44583)
0925 00:46:22.414688 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.414692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.415519 (+ 827us) server_negotiation.cc:407] Connection header received
0925 00:46:22.416946 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.416950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.417008 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.417133 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:22.418306 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.418950 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.420510 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.420901 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.421031 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.423473 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:22.423478 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:22.423482 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:22.423510 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:22.425067 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:22.425727 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:22.425730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:22.425731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:22.425846 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:22.426193 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:22.426198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:22.426200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:22.426422 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:22.426529 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.427367 (+ 838us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.427476 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 00:46:22.427974 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.419663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.6.72.126:44583)
0925 00:46:22.419895 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.419900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.420000 (+ 100us) server_negotiation.cc:407] Connection header received
0925 00:46:22.420092 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.420095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.420153 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.420261 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:22.421181 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.421907 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.422818 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.423122 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.423230 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.424852 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:22.424855 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:22.424857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:22.424888 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:22.426417 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:22.426778 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:22.426780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:22.426782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:22.426846 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:22.427138 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:22.427140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:22.427142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:22.427333 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:22.427525 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.427711 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.427841 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":66,"threads_started":1}
I20260925 00:46:22.428702 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.414882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:52733)
0925 00:46:22.415356 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:22.415399 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:22.416731 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:22.417311 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:22.417318 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:22.417685 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:46:22.418159 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.418172 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.419109 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.419116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.420395 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.420405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.421044 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.421047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.421231 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.421821 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:46:22.421839 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:46:22.423315 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:46:22.425174 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:22.425182 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:22.425198 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:22.425580 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:22.425951 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:22.425953 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:22.425955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:22.426091 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:22.426805 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:46:22.426810 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:46:22.427117 (+ 307us) client_negotiation.cc:769] Sending connection context
0925 00:46:22.427303 (+ 186us) client_negotiation.cc:240] Negotiation successful
0925 00:46:22.427663 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":117,"threads_started":1}
I20260925 00:46:22.429776 22817 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44583
I20260925 00:46:22.430027 22817 heartbeater.cc:464] Registering TS with master...
I20260925 00:46:22.430486 22817 heartbeater.cc:511] Master 127.6.72.126:44583 requested a full tablet report, sending...
I20260925 00:46:22.431419 22359 ts_manager.cc:194] Registered new tserver with Master: 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
I20260925 00:46:22.431917 22359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:52733
I20260925 00:46:22.441847 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:46:22.445684 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.442459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.6.72.126:44583)
0925 00:46:22.442620 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.442625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.442730 (+ 105us) server_negotiation.cc:407] Connection header received
0925 00:46:22.442879 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.442881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.442978 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.443061 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:46:22.443262 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.443777 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.444638 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.444866 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.444942 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.445204 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:46:22.445334 (+ 130us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 00:46:22.445535 (+ 201us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34702: token expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260925 00:46:22.445760 22821 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34702: token expired
W20260925 00:46:22.446074 22285 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.72.126:44583)
I20260925 00:46:22.453222 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.446416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34706 (local address 127.6.72.126:44583)
0925 00:46:22.446573 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.446577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.446590 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:46:22.446661 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.446665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.446710 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.446793 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:22.447398 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.447884 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.448732 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.448943 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.449055 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.450332 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:22.450336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:22.450338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:22.450368 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:22.451545 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:22.451958 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:22.451962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:22.451964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:22.452046 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:22.452341 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:22.452345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:22.452347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:22.452558 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:22.452699 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.452947 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.453061 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 00:46:22.455708 22359 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:46:22.459089 22359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:46:22.466806 22481 tablet_service.cc:1467] Processing CreateTablet for tablet bc42f26a31ed47af89d850c049cf9a81 (DEFAULT_TABLE table=test-table [id=7844f172a5674f728a150416010d9a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:46:22.467450 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc42f26a31ed47af89d850c049cf9a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:46:22.470156 22439 tablet_bootstrap.cc:492] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Bootstrap starting.
I20260925 00:46:22.471097 22439 tablet_bootstrap.cc:654] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Neither blocks nor log segments found. Creating new log.
I20260925 00:46:22.471305 22553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.465688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:44913 (local address 127.0.0.1:32890)
0925 00:46:22.465812 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:22.465830 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:22.465911 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:22.466921 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:22.466924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:22.466943 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:46:22.467054 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.467061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.468527 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.468531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.469722 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.469732 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.470849 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.470853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.470942 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.470978 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 00:46:22.471078 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 00:46:22.471169 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 00:46:22.471369 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.465902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32890 (local address 127.6.72.66:44913)
0925 00:46:22.466381 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.466396 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.466422 (+ 26us) server_negotiation.cc:407] Connection header received
0925 00:46:22.466510 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.466516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.466756 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.466943 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:46:22.467162 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.468395 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.469876 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.470716 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.470835 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.470990 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.471109 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.471210 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":156,"threads_started":1}
I20260925 00:46:22.471724 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.465996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:38415 (local address 127.0.0.1:60348)
0925 00:46:22.466924 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:22.466938 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:22.467031 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:22.467556 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:22.467559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:22.467577 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:46:22.467664 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.467670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.468981 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.468984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.470094 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.470103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.471307 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.471311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.471407 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.471433 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 00:46:22.471507 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 00:46:22.471594 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"thread_start_us":105,"threads_started":1}
I20260925 00:46:22.472146 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.466334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.6.72.67:38415)
0925 00:46:22.466740 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.466752 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.467125 (+ 373us) server_negotiation.cc:407] Connection header received
0925 00:46:22.467211 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.467216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.467399 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.467567 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:46:22.467748 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.468851 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.470224 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.471013 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.471116 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.471244 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.471874 (+ 630us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.471983 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":176,"threads_started":1}
I20260925 00:46:22.472502 22439 tablet_bootstrap.cc:492] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: No bootstrap required, opened a new log
I20260925 00:46:22.472597 22439 ts_tablet_manager.cc:1397] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 00:46:22.473320 22439 raft_consensus.cc:348] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.473431 22439 raft_consensus.cc:374] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:46:22.473467 22439 raft_consensus.cc:729] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac4e51acafe84433a401f3df893bda8f, State: Initialized, Role: FOLLOWER
I20260925 00:46:22.473600 22439 consensus_queue.cc:260] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.473850 22439 ts_tablet_manager.cc:1428] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 00:46:22.474987 22751 tablet_service.cc:1467] Processing CreateTablet for tablet bc42f26a31ed47af89d850c049cf9a81 (DEFAULT_TABLE table=test-table [id=7844f172a5674f728a150416010d9a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:46:22.475057 22619 tablet_service.cc:1467] Processing CreateTablet for tablet bc42f26a31ed47af89d850c049cf9a81 (DEFAULT_TABLE table=test-table [id=7844f172a5674f728a150416010d9a7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:46:22.476162 22619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc42f26a31ed47af89d850c049cf9a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:46:22.476184 22751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc42f26a31ed47af89d850c049cf9a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:46:22.482971 22830 tablet_bootstrap.cc:492] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: Bootstrap starting.
I20260925 00:46:22.483239 22831 tablet_bootstrap.cc:492] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: Bootstrap starting.
I20260925 00:46:22.485306 22831 tablet_bootstrap.cc:654] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: Neither blocks nor log segments found. Creating new log.
I20260925 00:46:22.485692 22830 tablet_bootstrap.cc:654] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: Neither blocks nor log segments found. Creating new log.
I20260925 00:46:22.486361 22831 log.cc:824] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: Log is configured to *not* fsync() on all Append() calls
I20260925 00:46:22.486361 22830 log.cc:824] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: Log is configured to *not* fsync() on all Append() calls
I20260925 00:46:22.488209 22830 tablet_bootstrap.cc:492] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: No bootstrap required, opened a new log
I20260925 00:46:22.488363 22831 tablet_bootstrap.cc:492] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: No bootstrap required, opened a new log
I20260925 00:46:22.488464 22830 ts_tablet_manager.cc:1397] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260925 00:46:22.488575 22831 ts_tablet_manager.cc:1397] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260925 00:46:22.491674 22830 raft_consensus.cc:348] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.491688 22831 raft_consensus.cc:348] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.491899 22830 raft_consensus.cc:374] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:46:22.491928 22831 raft_consensus.cc:374] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:46:22.491956 22830 raft_consensus.cc:729] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cc60a698bd14fea84ed074626d01996, State: Initialized, Role: FOLLOWER
I20260925 00:46:22.491990 22831 raft_consensus.cc:729] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bbe5b35aa2b4768b1713ed802bafe21, State: Initialized, Role: FOLLOWER
I20260925 00:46:22.492441 22831 consensus_queue.cc:260] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.492429 22830 consensus_queue.cc:260] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.493083 22817 heartbeater.cc:503] Master 127.6.72.126:44583 was elected leader, sending a full tablet report...
I20260925 00:46:22.493427 22830 ts_tablet_manager.cc:1428] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260925 00:46:22.493458 22831 ts_tablet_manager.cc:1428] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260925 00:46:22.494300 22685 heartbeater.cc:503] Master 127.6.72.126:44583 was elected leader, sending a full tablet report...
W20260925 00:46:22.530615 22548 tablet.cc:2369] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:46:22.627707 22555 raft_consensus.cc:482] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:46:22.627946 22555 raft_consensus.cc:504] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.629139 22555 leader_election.cc:288] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913), 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
I20260925 00:46:22.633257 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.629588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:48451 (local address 127.6.72.66:44913)
0925 00:46:22.629733 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.629737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.629760 (+ 23us) server_negotiation.cc:407] Connection header received
0925 00:46:22.629975 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.629979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.630042 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.630158 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:46:22.630464 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.630993 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.632137 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.632639 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.632776 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.632849 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.633034 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.633111 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 00:46:22.633291 22836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.629404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:44913 (local address 127.6.72.65:48451)
0925 00:46:22.629695 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:22.629712 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:22.629818 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:22.630172 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:22.630175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:22.630197 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:46:22.630326 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.630332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.631123 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.631129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.631972 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.631982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.632778 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.632782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.632880 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.632911 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 00:46:22.633018 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 00:46:22.633111 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":110,"threads_started":1}
I20260925 00:46:22.633342 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.630014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:41651 (local address 127.6.72.67:38415)
0925 00:46:22.630183 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.630188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.630303 (+ 115us) server_negotiation.cc:407] Connection header received
0925 00:46:22.630413 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.630416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.630468 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.630558 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:46:22.630824 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.631316 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.632365 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.632769 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.632896 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.632966 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.633154 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.633224 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 00:46:22.633343 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.629823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:38415 (local address 127.6.72.65:41651)
0925 00:46:22.630212 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:22.630224 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:22.630322 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:22.630571 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:22.630574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:22.630590 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:46:22.630709 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.630714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.631447 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.631451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.632186 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:22.632194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.632900 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.632903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:22.633008 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.633029 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 00:46:22.633124 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 00:46:22.633198 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":97,"threads_started":1}
I20260925 00:46:22.634039 22771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc42f26a31ed47af89d850c049cf9a81" candidate_uuid: "ac4e51acafe84433a401f3df893bda8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" is_pre_election: true
I20260925 00:46:22.634166 22639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc42f26a31ed47af89d850c049cf9a81" candidate_uuid: "ac4e51acafe84433a401f3df893bda8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc60a698bd14fea84ed074626d01996" is_pre_election: true
I20260925 00:46:22.634383 22771 raft_consensus.cc:2454] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac4e51acafe84433a401f3df893bda8f in term 0.
I20260925 00:46:22.634383 22639 raft_consensus.cc:2454] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac4e51acafe84433a401f3df893bda8f in term 0.
I20260925 00:46:22.634892 22436 leader_election.cc:302] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc60a698bd14fea84ed074626d01996, ac4e51acafe84433a401f3df893bda8f; no voters:
I20260925 00:46:22.635202 22555 raft_consensus.cc:2781] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:46:22.635290 22555 raft_consensus.cc:482] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:46:22.635322 22555 raft_consensus.cc:3037] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:46:22.636559 22555 raft_consensus.cc:504] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.636930 22555 leader_election.cc:288] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 1 election: Requested vote from peers 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913), 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
I20260925 00:46:22.637424 22639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc42f26a31ed47af89d850c049cf9a81" candidate_uuid: "ac4e51acafe84433a401f3df893bda8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cc60a698bd14fea84ed074626d01996"
I20260925 00:46:22.637573 22771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc42f26a31ed47af89d850c049cf9a81" candidate_uuid: "ac4e51acafe84433a401f3df893bda8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bbe5b35aa2b4768b1713ed802bafe21"
I20260925 00:46:22.637563 22639 raft_consensus.cc:3037] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:46:22.637746 22771 raft_consensus.cc:3037] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:46:22.638677 22639 raft_consensus.cc:2454] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac4e51acafe84433a401f3df893bda8f in term 1.
I20260925 00:46:22.638888 22771 raft_consensus.cc:2454] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac4e51acafe84433a401f3df893bda8f in term 1.
I20260925 00:46:22.639055 22436 leader_election.cc:302] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cc60a698bd14fea84ed074626d01996, ac4e51acafe84433a401f3df893bda8f; no voters:
I20260925 00:46:22.639341 22555 raft_consensus.cc:2781] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:46:22.639539 22555 raft_consensus.cc:686] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 LEADER]: Becoming Leader. State: Replica: ac4e51acafe84433a401f3df893bda8f, State: Running, Role: LEADER
I20260925 00:46:22.639802 22555 consensus_queue.cc:237] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:22.643107 22359 catalog_manager.cc:5495] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ac4e51acafe84433a401f3df893bda8f (127.6.72.65). New cstate: current_term: 1 leader_uuid: "ac4e51acafe84433a401f3df893bda8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } health_report { overall_health: HEALTHY } } }
W20260925 00:46:22.663627 22818 tablet.cc:2369] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:46:22.664445 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.660338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.6.72.65:35201)
0925 00:46:22.660688 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:22.660695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:22.660708 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:46:22.660784 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:22.660787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:22.660840 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:22.660984 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:46:22.661236 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.661802 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.662647 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:22.662887 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:22.663013 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:22.663318 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:46:22.663959 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:46:22.664077 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:22.664164 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 00:46:22.664232 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":108,"threads_started":1}
I20260925 00:46:22.668875 22639 raft_consensus.cc:1261] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996 [term 1 FOLLOWER]: Refusing update from remote peer ac4e51acafe84433a401f3df893bda8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:46:22.668905 22771 raft_consensus.cc:1261] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 1 FOLLOWER]: Refusing update from remote peer ac4e51acafe84433a401f3df893bda8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:46:22.669624 22838 consensus_queue.cc:1035] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:46:22.669849 22555 consensus_queue.cc:1035] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:46:22.678561 22843 mvcc.cc:205] Tried to move back new op lower bound from 7333057260204834816 to 7333057260095614976. Current Snapshot: MvccSnapshot[applied={T|T < 7333057260204834816}]
I20260925 00:46:22.686595 22359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:46:22.686802 22359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:46:22.689472 22359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Sending DeleteTablet for 3 replicas of tablet bc42f26a31ed47af89d850c049cf9a81
I20260925 00:46:22.690222 22619 tablet_service.cc:1514] Processing DeleteTablet for tablet bc42f26a31ed47af89d850c049cf9a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:46:22 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:32890
I20260925 00:46:22.690295 22751 tablet_service.cc:1514] Processing DeleteTablet for tablet bc42f26a31ed47af89d850c049cf9a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:46:22 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:60348
I20260925 00:46:22.690739 22481 tablet_service.cc:1514] Processing DeleteTablet for tablet bc42f26a31ed47af89d850c049cf9a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:46:22 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:47646
I20260925 00:46:22.690769 22851 tablet_replica.cc:330] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: stopping tablet replica
I20260925 00:46:22.690996 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 22397
I20260925 00:46:22.692857 22852 tablet_replica.cc:330] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: stopping tablet replica
I20260925 00:46:22.693169 22851 raft_consensus.cc:2229] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:46:22.693169 22852 raft_consensus.cc:2229] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:46:22.693531 22851 raft_consensus.cc:2258] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:46:22.693531 22852 raft_consensus.cc:2258] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:46:22.695346 22852 ts_tablet_manager.cc:1905] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:46:22.697471 22851 ts_tablet_manager.cc:1905] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:46:22.699128 22852 ts_tablet_manager.cc:1918] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:46:22.699245 22852 log.cc:1196] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/bc42f26a31ed47af89d850c049cf9a81
I20260925 00:46:22.699646 22852 ts_tablet_manager.cc:1939] T bc42f26a31ed47af89d850c049cf9a81 P 9bbe5b35aa2b4768b1713ed802bafe21: Deleting consensus metadata
W20260925 00:46:22.699633 22344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:35201 (error 108)
I20260925 00:46:22.699859 22851 ts_tablet_manager.cc:1918] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:46:22.699972 22851 log.cc:1196] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/bc42f26a31ed47af89d850c049cf9a81
I20260925 00:46:22.700342 22851 ts_tablet_manager.cc:1939] T bc42f26a31ed47af89d850c049cf9a81 P 6cc60a698bd14fea84ed074626d01996: Deleting consensus metadata
I20260925 00:46:22.701030 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.700738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:35201 (local address 127.0.0.1:47656)
0925 00:46:22.700859 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:22.700934 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:35201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 00:46:22.701081 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 22551
W20260925 00:46:22.701802 22344 catalog_manager.cc:4584] TS ac4e51acafe84433a401f3df893bda8f (127.6.72.65:35201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc42f26a31ed47af89d850c049cf9a81: Network error: Client connection negotiation failed: client connection to 127.6.72.65:35201: connect: Connection refused (error 111)
I20260925 00:46:22.702656 22346 catalog_manager.cc:4841] TS 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415): tablet bc42f26a31ed47af89d850c049cf9a81 (table test-table [id=7844f172a5674f728a150416010d9a7c]) successfully deleted
I20260925 00:46:22.706657 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 22688
I20260925 00:46:22.707288 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:22.706896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:44913 (local address 127.0.0.1:32894)
0925 00:46:22.707118 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:22.707180 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.66:44913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260925 00:46:22.712738 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 22326
I20260925 00:46:42.719767 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44583
--webserver_interface=127.6.72.126
--webserver_port=37023
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:44583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:46:42.832281 22855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:42.832583 22855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:42.832659 22855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:42.836771 22855 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 00:46:42.836820 22855 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 00:46:42.836836 22855 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 00:46:42.836850 22855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:46:42.836894 22855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:42.836930 22855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:42.836961 22855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:46:42.836993 22855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:46:42.842362 22855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:44583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=37023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:42.843456 22855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:42.844522 22855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 00:46:42.853617 22855 server_base.cc:1034] running on GCE node
W20260925 00:46:42.853554 22860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:42.853637 22863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:42.853559 22861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:42.854409 22855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:42.855659 22855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:42.856824 22855 hybrid_clock.cc:648] HybridClock initialized: now 1790297202856804 us; error 37 us; skew 500 ppm
Sep 25 00:46:42 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297202, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:42.859730 22855 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:46:42.860778 22855 webserver.cc:460] Webserver started at http://127.6.72.126:37023/ using document root <none> and password file <none>
I20260925 00:46:42.861310 22855 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:42.861370 22855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:42.865315 22855 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:46:42.867561 22870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:42.869014 22855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 00:46:42.869164 22855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "9fec82d8b0514734a1f04f0c624887f8"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "3cb753e71ea13b5fa5a8c7b65c7fb7bd"
server_key_iv: "b0b0b94f03e36204b881a7d5a09220d5"
server_key_version: "encryptionkey@0"
I20260925 00:46:42.869763 22855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:42.893464 22855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:42.896615 22855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:42.896832 22855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:42.904307 22855 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:44583
I20260925 00:46:42.904692 22922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:44583 every 8 connection(s)
I20260925 00:46:42.905442 22855 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:46:42.906837 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 22855
I20260925 00:46:42.907274 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:35201
--local_ip_for_outbound_sockets=127.6.72.65
--tserver_master_addrs=127.6.72.126:44583
--webserver_port=42835
--webserver_interface=127.6.72.65
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:46:42.910482 22923 sys_catalog.cc:263] Verifying existing consensus state
I20260925 00:46:42.913226 22923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Bootstrap starting.
I20260925 00:46:42.937000 22923 log.cc:824] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Log is configured to *not* fsync() on all Append() calls
I20260925 00:46:42.951025 22923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:46:42.951558 22923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: Bootstrap complete.
I20260925 00:46:42.957199 22923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:42.957731 22923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fec82d8b0514734a1f04f0c624887f8, State: Initialized, Role: FOLLOWER
I20260925 00:46:42.958283 22923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:42.958441 22923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:46:42.958511 22923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:46:42.958596 22923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 2 FOLLOWER]: Advancing to term 3
I20260925 00:46:42.961143 22923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:42.961498 22923 leader_election.cc:302] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fec82d8b0514734a1f04f0c624887f8; no voters:
I20260925 00:46:42.961836 22923 leader_election.cc:288] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 00:46:42.961994 22927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 00:46:42.962435 22927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [term 3 LEADER]: Becoming Leader. State: Replica: 9fec82d8b0514734a1f04f0c624887f8, State: Running, Role: LEADER
I20260925 00:46:42.962805 22927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } }
I20260925 00:46:42.963137 22923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:46:42.964403 22929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fec82d8b0514734a1f04f0c624887f8. Latest consensus state: current_term: 3 leader_uuid: "9fec82d8b0514734a1f04f0c624887f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } } }
I20260925 00:46:42.964535 22929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: This master's current role is: LEADER
I20260925 00:46:42.964214 22928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9fec82d8b0514734a1f04f0c624887f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fec82d8b0514734a1f04f0c624887f8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44583 } } }
I20260925 00:46:42.964947 22928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8 [sys.catalog]: This master's current role is: LEADER
I20260925 00:46:42.968713 22944 catalog_manager.cc:1259] Loaded cluster ID: 11538457c3c24bc7b36aefb3e27a3991
I20260925 00:46:42.968783 22944 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: loading cluster ID for follower catalog manager: success
I20260925 00:46:42.971450 22944 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9fec82d8b0514734a1f04f0c624887f8: acquiring CA information for follower catalog manager: success
F20260925 00:46:42.972620 22944 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc3b68339ad google::LogMessage::Fail() at ??:0
@ 0x7fc3b683688a google::LogMessage::SendToLog() at ??:0
@ 0x7fc3b68334de google::LogMessage::Flush() at ??:0
@ 0x7fc3b6834a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc3b94f718d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc3c1198989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7fc3c1175023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7fc3c117554b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7fc3c116e652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7fc3c116d9e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7fc3c11b4541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc3b8a76512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc3ba0ba6db start_thread at ??:0
@ 0x7fc3b533161f clone at ??:0
W20260925 00:46:43.060144 22925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:43.060513 22925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:43.060597 22925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:43.064765 22925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:43.064843 22925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:43.064932 22925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:46:43.070299 22925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:35201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=42835
--tserver_master_addrs=127.6.72.126:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:43.071369 22925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:43.072455 22925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:43.082063 22965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:43.082261 22925 server_base.cc:1034] running on GCE node
W20260925 00:46:43.082110 22964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:43.082062 22967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:43.082935 22925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:43.083575 22925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:43.084755 22925 hybrid_clock.cc:648] HybridClock initialized: now 1790297203084733 us; error 39 us; skew 500 ppm
Sep 25 00:46:43 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297203, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:43.087697 22925 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:46:43.088744 22925 webserver.cc:460] Webserver started at http://127.6.72.65:42835/ using document root <none> and password file <none>
I20260925 00:46:43.089310 22925 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:43.089399 22925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:43.093252 22925 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:46:43.095808 22974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:43.097169 22925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:46:43.097302 22925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "ac4e51acafe84433a401f3df893bda8f"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "c9ccb94c928e61560a53306f879aab5d"
server_key_iv: "64c075a056374f31ee5cea851410b1ea"
server_key_version: "encryptionkey@0"
I20260925 00:46:43.097852 22925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:43.124233 22925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:43.128989 22925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:43.129318 22925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:43.130514 22925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:46:43.132041 22981 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 00:46:43.137655 22925 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 00:46:43.137732 22925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260925 00:46:43.137814 22925 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 00:46:43.139547 22925 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 00:46:43.139626 22925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260925 00:46:43.139909 22981 tablet_bootstrap.cc:492] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Bootstrap starting.
I20260925 00:46:43.150218 22925 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:35201
I20260925 00:46:43.150278 23088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:35201 every 8 connection(s)
I20260925 00:46:43.151631 22925 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:46:43.155668 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 22925
I20260925 00:46:43.156278 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:44913
--local_ip_for_outbound_sockets=127.6.72.66
--tserver_master_addrs=127.6.72.126:44583
--webserver_port=33079
--webserver_interface=127.6.72.66
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:46:43.161036 22981 log.cc:824] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Log is configured to *not* fsync() on all Append() calls
I20260925 00:46:43.171442 22981 tablet_bootstrap.cc:492] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 00:46:43.171890 22981 tablet_bootstrap.cc:492] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Bootstrap complete.
I20260925 00:46:43.173317 22981 ts_tablet_manager.cc:1397] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.007s
I20260925 00:46:43.176671 22981 raft_consensus.cc:348] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:43.177222 22981 raft_consensus.cc:729] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac4e51acafe84433a401f3df893bda8f, State: Initialized, Role: FOLLOWER
I20260925 00:46:43.177740 22981 consensus_queue.cc:260] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:43.179050 22981 ts_tablet_manager.cc:1428] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260925 00:46:43.189872 23091 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:46:43.154037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:37461)
0925 00:46:43.154691 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:43.154751 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:43.155886 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:43.188972 (+ 33086us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":181,"threads_started":1}
W20260925 00:46:43.190507 23089 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:44583 (0 consecutive failures): Network error: Failed to ping master at 127.6.72.126:44583: Client connection negotiation failed: client connection to 127.6.72.126:44583: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 00:46:43.191248 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:43.190973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:36679)
0925 00:46:43.191105 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:43.191179 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 00:46:43.191926 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:43.191719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:55023)
0925 00:46:43.191824 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:43.191872 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 00:46:43.192114 23089 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 00:46:43.276147 23092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:43.276445 23092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:43.276494 23092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:43.280766 23092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:43.280825 23092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:43.280923 23092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:46:43.286231 23092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:44913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=33079
--tserver_master_addrs=127.6.72.126:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:43.287382 23092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:43.288472 23092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:43.298295 23102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:43.298432 23099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:43.298458 23100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:43.298678 23092 server_base.cc:1034] running on GCE node
I20260925 00:46:43.299180 23092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:43.299834 23092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:43.301040 23092 hybrid_clock.cc:648] HybridClock initialized: now 1790297203300998 us; error 44 us; skew 500 ppm
Sep 25 00:46:43 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297203, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:43.304384 23092 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:46:43.305321 23092 webserver.cc:460] Webserver started at http://127.6.72.66:33079/ using document root <none> and password file <none>
I20260925 00:46:43.305877 23092 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:43.305944 23092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:43.309998 23092 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:46:43.312146 23109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:43.313632 23092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:46:43.313777 23092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "6cc60a698bd14fea84ed074626d01996"
format_stamp: "Formatted at 2026-09-25 00:46:00 on dist-test-slave-w8nh"
server_key: "8fdb0b254905be185faa3c84c61cfb12"
server_key_iv: "76ace560419756b70618ec62e43c9c25"
server_key_version: "encryptionkey@0"
I20260925 00:46:43.314371 23092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:43.328806 23092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:43.332458 23092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:43.332661 23092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:43.333292 23092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:46:43.334231 23092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:46:43.334304 23092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:43.334380 23092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:46:43.334424 23092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:43.343587 23092 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:44913
I20260925 00:46:43.343641 23222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:44913 every 8 connection(s)
I20260925 00:46:43.344780 23092 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:46:43.348750 23225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:43.346991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:52087)
0925 00:46:43.347445 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:43.347548 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":114,"threads_started":1}
W20260925 00:46:43.349182 23223 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:44583 (0 consecutive failures): Network error: Failed to ping master at 127.6.72.126:44583: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
I20260925 00:46:43.349762 23225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:43.349563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:45001)
0925 00:46:43.349667 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:43.349712 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 00:46:43.350590 23225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:43.350182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:38981)
0925 00:46:43.350452 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:43.350542 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 00:46:43.350941 23223 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 00:46:43.354372 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 23092
I20260925 00:46:43.354802 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:38415
--local_ip_for_outbound_sockets=127.6.72.67
--tserver_master_addrs=127.6.72.126:44583
--webserver_port=35777
--webserver_interface=127.6.72.67
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:46:43.468587 23226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:43.468866 23226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:43.468912 23226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:43.473034 23226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:43.473107 23226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:43.473214 23226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:46:43.478794 23226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:38415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=35777
--tserver_master_addrs=127.6.72.126:44583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:43.479991 23226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:43.481117 23226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:43.491118 23234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:43.491129 23231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:43.491151 23232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:43.491469 23226 server_base.cc:1034] running on GCE node
I20260925 00:46:43.491933 23226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:43.492656 23226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:43.493819 23226 hybrid_clock.cc:648] HybridClock initialized: now 1790297203493798 us; error 31 us; skew 500 ppm
Sep 25 00:46:43 dist-test-slave-w8nh krb5kdc[21778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297203, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:43.496667 23226 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:46:43.497531 23226 webserver.cc:460] Webserver started at http://127.6.72.67:35777/ using document root <none> and password file <none>
I20260925 00:46:43.498099 23226 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:43.498189 23226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:43.502246 23226 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:46:43.504438 23241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:43.505728 23226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:46:43.505851 23226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "9bbe5b35aa2b4768b1713ed802bafe21"
format_stamp: "Formatted at 2026-09-25 00:46:01 on dist-test-slave-w8nh"
server_key: "8aca937fae95b889945c4ff2b73faaef"
server_key_iv: "75f7f58b0b524dd9d8c89b36e671255e"
server_key_version: "encryptionkey@0"
I20260925 00:46:43.506405 23226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:43.524832 23226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:43.528031 23226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:43.528316 23226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:43.529058 23226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:46:43.530069 23226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:46:43.530143 23226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:43.530217 23226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:46:43.530267 23226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:43.539939 23226 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:38415
I20260925 00:46:43.539958 23354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:38415 every 8 connection(s)
I20260925 00:46:43.541141 23226 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:46:43.541822 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 23226
I20260925 00:46:43.545293 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:43.543440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:39341)
0925 00:46:43.543941 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:43.544050 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":130,"threads_started":1}
W20260925 00:46:43.545782 23355 heartbeater.cc:644] Failed to heartbeat to 127.6.72.126:44583 (0 consecutive failures): Network error: Failed to ping master at 127.6.72.126:44583: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
I20260925 00:46:43.546449 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:43.546191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:59505)
0925 00:46:43.546333 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:43.546388 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 00:46:43.547191 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:43.546952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:36251)
0925 00:46:43.547078 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:43.547125 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 00:46:43.547416 23355 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 00:46:44.193236 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:44.192772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:34435)
0925 00:46:44.193051 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:44.193115 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":103,"threads_started":1}
I20260925 00:46:44.352118 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:44.351655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:58759)
0925 00:46:44.351950 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:44.352015 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":78,"threads_started":1}
I20260925 00:46:44.390518 23361 raft_consensus.cc:482] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:46:44.390831 23361 raft_consensus.cc:504] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:44.391932 23361 leader_election.cc:288] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913), 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
Sep 25 00:46:44 dist-test-slave-w8nh krb5kdc[21778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297203, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.66@KRBTEST.COM
Sep 25 00:46:44 dist-test-slave-w8nh krb5kdc[21778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297203, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.67@KRBTEST.COM
I20260925 00:46:44.406744 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:44.392607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:51695 (local address 127.6.72.66:44913)
0925 00:46:44.392977 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:44.393003 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:44.393049 (+ 46us) server_negotiation.cc:407] Connection header received
0925 00:46:44.393743 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:44.393749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:44.394087 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:44.394405 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:44.395724 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:44.396791 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:44.398009 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:44.398477 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:44.398682 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:44.401654 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:44.401663 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:44.401682 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:44.401721 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:44.403759 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:44.404444 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:44.404450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:44.404457 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:44.404608 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:44.405070 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:44.405076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:44.405079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:44.405501 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:44.405667 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:44.406364 (+ 697us) server_negotiation.cc:299] Negotiation successful
0925 00:46:44.406589 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":80,"threads_started":1}
I20260925 00:46:44.406872 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:44.392664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:38415 (local address 127.6.72.65:43099)
0925 00:46:44.392927 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:44.392940 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:44.393025 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:44.394621 (+ 1596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:44.394625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:44.395022 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:46:44.395512 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:44.395527 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:44.397033 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:44.397037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:44.397871 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:44.397880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:44.398653 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:44.398657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:44.398820 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:44.399666 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:46:44.399679 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:46:44.401716 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:46:44.404059 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:44.404062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:44.404063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:44.404436 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:44.404918 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:44.404920 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:44.404921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:44.405027 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:44.405730 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:46:44.405732 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:46:44.405961 (+ 229us) client_negotiation.cc:769] Sending connection context
0925 00:46:44.406248 (+ 287us) client_negotiation.cc:240] Negotiation successful
0925 00:46:44.406661 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":95,"spinlock_wait_cycles":36352,"thread_start_us":61,"threads_started":1}
I20260925 00:46:44.407192 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:44.392964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:43099 (local address 127.6.72.67:38415)
0925 00:46:44.393324 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:44.393335 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:44.393370 (+ 35us) server_negotiation.cc:407] Connection header received
0925 00:46:44.394033 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:44.394039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:44.394363 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:44.394692 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:44.395721 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:44.396876 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:44.398033 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:44.398512 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:44.398699 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:44.401897 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:44.401902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:44.401911 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:44.401938 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:44.403926 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:44.404591 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:44.404596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:44.404600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:44.404810 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:44.405131 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:44.405133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:44.405135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:44.405518 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:44.405684 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:44.406698 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0925 00:46:44.406988 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":157,"threads_started":1}
I20260925 00:46:44.407274 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:44.392338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:44913 (local address 127.6.72.65:51695)
0925 00:46:44.392505 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:44.392519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:46:44.392674 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:46:44.394455 (+ 1781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:46:44.394461 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:46:44.394995 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:46:44.395512 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:44.395526 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:44.396939 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:44.396943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:44.397869 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:46:44.397879 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:46:44.398614 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:46:44.398617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:46:44.398865 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:44.399600 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:46:44.399616 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:46:44.401318 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:46:44.403896 (+ 2578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:44.403902 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:44.403916 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:44.404321 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:44.404750 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:46:44.404754 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:46:44.404756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:46:44.404944 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:46:44.405649 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:46:44.405656 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:46:44.405961 (+ 305us) client_negotiation.cc:769] Sending connection context
0925 00:46:44.406248 (+ 287us) client_negotiation.cc:240] Negotiation successful
0925 00:46:44.407108 (+ 860us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":691,"spinlock_wait_cycles":185344}
W20260925 00:46:44.409940 22978 leader_election.cc:341] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913): Not found: Tablet not found: bc42f26a31ed47af89d850c049cf9a81
W20260925 00:46:44.410858 22978 leader_election.cc:341] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415): Not found: Tablet not found: bc42f26a31ed47af89d850c049cf9a81
I20260925 00:46:44.411005 22978 leader_election.cc:302] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac4e51acafe84433a401f3df893bda8f; no voters: 6cc60a698bd14fea84ed074626d01996, 9bbe5b35aa2b4768b1713ed802bafe21
I20260925 00:46:44.411334 23361 raft_consensus.cc:2726] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 00:46:44.548593 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:44.548101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:60405)
0925 00:46:44.548435 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:44.548495 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":75,"threads_started":1}
I20260925 00:46:45.194559 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:45.194072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:36993)
0925 00:46:45.194383 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:45.194449 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":88,"threads_started":1}
I20260925 00:46:45.353531 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:45.353016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:39129)
0925 00:46:45.353307 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:45.353398 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":84,"threads_started":1}
I20260925 00:46:45.549916 23368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:45.549476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:43885)
0925 00:46:45.549739 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:45.549802 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
I20260925 00:46:46.196122 23369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:46.195613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:35823)
0925 00:46:46.195953 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:46.196021 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":98,"threads_started":1}
I20260925 00:46:46.240739 23370 raft_consensus.cc:482] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:46:46.240876 23370 raft_consensus.cc:504] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:46.241261 23370 leader_election.cc:288] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913), 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
W20260925 00:46:46.242506 22978 leader_election.cc:341] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913): Not found: Tablet not found: bc42f26a31ed47af89d850c049cf9a81
W20260925 00:46:46.242662 22978 leader_election.cc:341] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415): Not found: Tablet not found: bc42f26a31ed47af89d850c049cf9a81
I20260925 00:46:46.242723 22978 leader_election.cc:302] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac4e51acafe84433a401f3df893bda8f; no voters: 6cc60a698bd14fea84ed074626d01996, 9bbe5b35aa2b4768b1713ed802bafe21
I20260925 00:46:46.242861 23370 raft_consensus.cc:2726] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 00:46:46.354797 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:46.354332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:44367)
0925 00:46:46.354606 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:46.354661 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":83,"threads_started":1}
I20260925 00:46:46.551273 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:46.550813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:32957)
0925 00:46:46.551110 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:46.551176 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":77,"threads_started":1}
I20260925 00:46:47.197731 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:47.197252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:57491)
0925 00:46:47.197560 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:47.197623 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":87,"threads_started":1}
I20260925 00:46:47.356483 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:47.355814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:51069)
0925 00:46:47.356300 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:47.356381 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":77,"threads_started":1}
I20260925 00:46:47.552796 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:47.552332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:51931)
0925 00:46:47.552606 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:47.552696 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":102,"threads_started":1}
I20260925 00:46:48.199316 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:48.198775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:39193)
0925 00:46:48.199147 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:48.199214 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":90,"threads_started":1}
I20260925 00:46:48.358179 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:48.357510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:55629)
0925 00:46:48.357973 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:48.358059 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":125,"threads_started":1}
I20260925 00:46:48.554230 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:48.553731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:33901)
0925 00:46:48.554058 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:48.554127 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":97,"threads_started":1}
I20260925 00:46:48.724421 23379 raft_consensus.cc:482] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:46:48.724548 23379 raft_consensus.cc:504] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:48.724960 23379 leader_election.cc:288] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913), 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
W20260925 00:46:48.725704 22978 leader_election.cc:341] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913): Not found: Tablet not found: bc42f26a31ed47af89d850c049cf9a81
W20260925 00:46:48.725854 22978 leader_election.cc:341] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415): Not found: Tablet not found: bc42f26a31ed47af89d850c049cf9a81
I20260925 00:46:48.725912 22978 leader_election.cc:302] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac4e51acafe84433a401f3df893bda8f; no voters: 6cc60a698bd14fea84ed074626d01996, 9bbe5b35aa2b4768b1713ed802bafe21
I20260925 00:46:48.726075 23379 raft_consensus.cc:2726] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 00:46:49.200729 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:49.200272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:48779)
0925 00:46:49.200567 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:49.200633 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":88,"threads_started":1}
I20260925 00:46:49.359551 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:49.359098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:49725)
0925 00:46:49.359377 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:49.359449 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":78,"threads_started":1}
I20260925 00:46:49.555657 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:49.555162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:55987)
0925 00:46:49.555439 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:49.555497 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":89,"threads_started":1}
I20260925 00:46:50.202117 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:50.201574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:50657)
0925 00:46:50.201929 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:50.202015 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":150,"threads_started":1}
I20260925 00:46:50.361006 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:50.360451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:47527)
0925 00:46:50.360828 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:50.360891 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":128,"threads_started":1}
I20260925 00:46:50.557111 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:50.556601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:33409)
0925 00:46:50.556948 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:50.557014 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":93,"threads_started":1}
I20260925 00:46:51.203465 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:51.202978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:36015)
0925 00:46:51.203296 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:51.203364 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":94,"threads_started":1}
I20260925 00:46:51.362613 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:51.361963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:50543)
0925 00:46:51.362376 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:51.362465 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":115,"threads_started":1}
I20260925 00:46:51.558399 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:51.557928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:46709)
0925 00:46:51.558232 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:51.558299 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":102,"threads_started":1}
I20260925 00:46:51.655380 23389 raft_consensus.cc:482] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:46:51.655521 23389 raft_consensus.cc:504] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:51.655977 23389 leader_election.cc:288] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913), 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
W20260925 00:46:51.657060 22978 leader_election.cc:341] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913): Not found: Tablet not found: bc42f26a31ed47af89d850c049cf9a81
W20260925 00:46:51.657179 22978 leader_election.cc:341] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415): Not found: Tablet not found: bc42f26a31ed47af89d850c049cf9a81
I20260925 00:46:51.657251 22978 leader_election.cc:302] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac4e51acafe84433a401f3df893bda8f; no voters: 6cc60a698bd14fea84ed074626d01996, 9bbe5b35aa2b4768b1713ed802bafe21
I20260925 00:46:51.657472 23389 raft_consensus.cc:2726] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 00:46:52.204938 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:52.204428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:48941)
0925 00:46:52.204763 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:52.204834 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":86,"threads_started":1}
I20260925 00:46:52.364080 23391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:52.363556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:43151)
0925 00:46:52.363895 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:52.363965 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":90,"threads_started":1}
I20260925 00:46:52.559893 23392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:52.559372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:42505)
0925 00:46:52.559711 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:52.559785 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":86,"threads_started":1}
I20260925 00:46:53.206633 23393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:53.206094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:35499)
0925 00:46:53.206451 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:53.206523 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":84,"threads_started":1}
I20260925 00:46:53.365573 23394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:53.365094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:36267)
0925 00:46:53.365399 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:53.365465 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":110,"threads_started":1}
I20260925 00:46:53.561353 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:53.560879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:54083)
0925 00:46:53.561174 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:53.561251 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":86,"threads_started":1}
I20260925 00:46:54.208355 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:54.207813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:60267)
0925 00:46:54.208151 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:54.208220 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":96,"threads_started":1}
I20260925 00:46:54.366931 23397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:54.366441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:58123)
0925 00:46:54.366765 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:54.366830 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
I20260925 00:46:54.563110 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:54.562452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:48931)
0925 00:46:54.562877 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:54.562971 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":158,"threads_started":1}
I20260925 00:46:55.209801 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:55.209235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:42575)
0925 00:46:55.209527 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:55.209680 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":79,"threads_started":1}
I20260925 00:46:55.368443 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:55.367823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:48915)
0925 00:46:55.368119 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:55.368286 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260925 00:46:55.564543 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:55.563989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:36213)
0925 00:46:55.564361 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:55.564431 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":82,"threads_started":1}
I20260925 00:46:56.211133 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:56.210710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:35857)
0925 00:46:56.210970 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:56.211029 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":73,"threads_started":1}
I20260925 00:46:56.369864 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:56.369386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:34713)
0925 00:46:56.369695 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:56.369763 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":90,"threads_started":1}
I20260925 00:46:56.565812 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:56.565375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:50921)
0925 00:46:56.565646 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:56.565710 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260925 00:46:56.816965 23405 raft_consensus.cc:482] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:46:56.817145 23405 raft_consensus.cc:504] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cc60a698bd14fea84ed074626d01996" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 44913 } } peers { permanent_uuid: "9bbe5b35aa2b4768b1713ed802bafe21" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 38415 } } peers { permanent_uuid: "ac4e51acafe84433a401f3df893bda8f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 35201 } }
I20260925 00:46:56.817679 23405 leader_election.cc:288] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913), 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415)
W20260925 00:46:56.818605 22978 leader_election.cc:341] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cc60a698bd14fea84ed074626d01996 (127.6.72.66:44913): Not found: Tablet not found: bc42f26a31ed47af89d850c049cf9a81
W20260925 00:46:56.818800 22978 leader_election.cc:341] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9bbe5b35aa2b4768b1713ed802bafe21 (127.6.72.67:38415): Not found: Tablet not found: bc42f26a31ed47af89d850c049cf9a81
I20260925 00:46:56.818861 22978 leader_election.cc:302] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac4e51acafe84433a401f3df893bda8f; no voters: 6cc60a698bd14fea84ed074626d01996, 9bbe5b35aa2b4768b1713ed802bafe21
I20260925 00:46:56.819098 23405 raft_consensus.cc:2726] T bc42f26a31ed47af89d850c049cf9a81 P ac4e51acafe84433a401f3df893bda8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 00:46:57.212352 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:57.211880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:34493)
0925 00:46:57.212136 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:57.212190 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":80,"threads_started":1}
I20260925 00:46:57.371085 23407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:57.370682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:47273)
0925 00:46:57.370930 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:57.370989 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
I20260925 00:46:57.567019 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:57.566571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.67:48401)
0925 00:46:57.566855 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:57.566922 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":120,"threads_started":1}
I20260925 00:46:58.213634 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:58.213255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.65:57565)
0925 00:46:58.213486 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:58.213537 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":75,"threads_started":1}
I20260925 00:46:58.372640 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:58.372142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44583 (local address 127.6.72.66:59115)
0925 00:46:58.372466 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0925 00:46:58.372539 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.126:44583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":122,"threads_started":1}
I20260925 00:46:58.553002 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 22925
I20260925 00:46:58.559043 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 23092
I20260925 00:46:58.564517 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 23226
2026-09-25T00:46:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59623 ms)
[----------] 30 tests from SecurityITest (247201 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:46:58 dist-test-slave-w8nh krb5kdc[23413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:46:58 dist-test-slave-w8nh krb5kdc[23413](info): set up 2 sockets
Sep 25 00:46:58 dist-test-slave-w8nh krb5kdc[23413](info): commencing operation
krb5kdc: starting...
W20260925 00:46:59.636214 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:46:59 dist-test-slave-w8nh krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:46:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:46:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:46:59.759904 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44967
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:40129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:44967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:46:59.871495 23426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:46:59.871819 23426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:46:59.871905 23426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:46:59.876164 23426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:46:59.876281 23426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:46:59.876348 23426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:46:59.876405 23426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:46:59.876443 23426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:46:59.881949 23426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:44967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:46:59.883080 23426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:46:59.884107 23426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:46:59.892719 23432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:59.892719 23431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:46:59.892824 23434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:46:59.892881 23426 server_base.cc:1034] running on GCE node
I20260925 00:46:59.893582 23426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:46:59.894661 23426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:46:59.895856 23426 hybrid_clock.cc:648] HybridClock initialized: now 1790297219895839 us; error 33 us; skew 500 ppm
Sep 25 00:46:59 dist-test-slave-w8nh krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297219, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:46:59.898706 23426 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:46:59.899600 23426 webserver.cc:460] Webserver started at http://127.6.72.126:37429/ using document root <none> and password file <none>
I20260925 00:46:59.900167 23426 fs_manager.cc:359] Metadata directory not provided
I20260925 00:46:59.900295 23426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:46:59.900537 23426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:46:59.902565 23426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "1c30b77638a447439db274976733974a"
format_stamp: "Formatted at 2026-09-25 00:46:59 on dist-test-slave-w8nh"
server_key: "04afd6a0edde7c094096182de100e894"
server_key_iv: "22fcb3a2f49d1b11a3fffbb8004e9a2e"
server_key_version: "encryptionkey@0"
I20260925 00:46:59.903142 23426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "1c30b77638a447439db274976733974a"
format_stamp: "Formatted at 2026-09-25 00:46:59 on dist-test-slave-w8nh"
server_key: "04afd6a0edde7c094096182de100e894"
server_key_iv: "22fcb3a2f49d1b11a3fffbb8004e9a2e"
server_key_version: "encryptionkey@0"
I20260925 00:46:59.906858 23426 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260925 00:46:59.909291 23441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:46:59.910601 23426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:46:59.910749 23426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "1c30b77638a447439db274976733974a"
format_stamp: "Formatted at 2026-09-25 00:46:59 on dist-test-slave-w8nh"
server_key: "04afd6a0edde7c094096182de100e894"
server_key_iv: "22fcb3a2f49d1b11a3fffbb8004e9a2e"
server_key_version: "encryptionkey@0"
I20260925 00:46:59.910867 23426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:46:59.933530 23426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:46:59.936946 23426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:46:59.937162 23426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:46:59.944499 23426 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:44967
I20260925 00:46:59.944527 23493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:44967 every 8 connection(s)
I20260925 00:46:59.945487 23426 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:46:59.946929 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 23426
I20260925 00:46:59.947068 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:46:59.947461 6433 external_mini_cluster.cc:1428] Setting key 2e85fc8ac7f456236abc3207cb2ac2be
I20260925 00:46:59.948702 23494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:46:59.954540 23494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a: Bootstrap starting.
Sep 25 00:46:59 dist-test-slave-w8nh krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:46:59.956969 23494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a: Neither blocks nor log segments found. Creating new log.
I20260925 00:46:59.957796 23494 log.cc:824] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a: Log is configured to *not* fsync() on all Append() calls
I20260925 00:46:59.959689 23494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a: No bootstrap required, opened a new log
I20260925 00:46:59.961535 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:46:59.948963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57948 (local address 127.6.72.126:44967)
0925 00:46:59.949421 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0925 00:46:59.949440 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 00:46:59.949475 (+ 35us) server_negotiation.cc:407] Connection header received
0925 00:46:59.950210 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:46:59.950226 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:46:59.950555 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:46:59.950897 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:46:59.951710 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:59.952820 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:59.953814 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:46:59.954290 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:46:59.954462 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:46:59.957035 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:46:59.957040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:46:59.957053 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:46:59.957106 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:46:59.958730 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:59.959184 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:59.959189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:59.959193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:59.959296 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:46:59.959583 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:46:59.959586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:46:59.959587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:46:59.960029 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:46:59.960233 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 00:46:59.960565 (+ 332us) server_negotiation.cc:299] Negotiation successful
0925 00:46:59.960845 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":164,"threads_started":1}
I20260925 00:46:59.963361 23494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c30b77638a447439db274976733974a" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44967 } }
I20260925 00:46:59.963719 23494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:46:59.963793 23494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c30b77638a447439db274976733974a, State: Initialized, Role: FOLLOWER
I20260925 00:46:59.964167 23494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c30b77638a447439db274976733974a" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44967 } }
I20260925 00:46:59.964350 23494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:46:59.964401 23494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:46:59.964473 23494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:46:59.965342 23494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c30b77638a447439db274976733974a" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44967 } }
I20260925 00:46:59.965644 23494 leader_election.cc:302] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c30b77638a447439db274976733974a; no voters:
I20260925 00:46:59.965899 23494 leader_election.cc:288] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:46:59.966038 23499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:46:59.966297 23499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [term 1 LEADER]: Becoming Leader. State: Replica: 1c30b77638a447439db274976733974a, State: Running, Role: LEADER
I20260925 00:46:59.966620 23499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c30b77638a447439db274976733974a" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44967 } }
I20260925 00:46:59.967098 23494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:46:59.968341 23501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c30b77638a447439db274976733974a. Latest consensus state: current_term: 1 leader_uuid: "1c30b77638a447439db274976733974a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c30b77638a447439db274976733974a" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44967 } } }
I20260925 00:46:59.968474 23501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [sys.catalog]: This master's current role is: LEADER
I20260925 00:46:59.968768 23500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c30b77638a447439db274976733974a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c30b77638a447439db274976733974a" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44967 } } }
I20260925 00:46:59.969043 23500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a [sys.catalog]: This master's current role is: LEADER
W20260925 00:46:59.972018 23514 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:46:59.972087 23514 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:46:59.972203 23508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:46:59.973003 23508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:46:59.978824 23508 catalog_manager.cc:1347] Generated new cluster ID: ba75e248ef9b45ef96cd674fbcd5a6f7
I20260925 00:46:59.978910 23508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:00.007457 23508 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:00.008848 23508 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:00.020545 23508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a: Generated new TSK 0
I20260925 00:47:00.021446 23508 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:47:00.059099 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44967
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:40129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:00.173044 23520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:00.173302 23520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:00.173379 23520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:00.177476 23520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:00.177533 23520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:00.177592 23520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:47:00.182771 23520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:00.183805 23520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:00.184907 23520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:00.194999 23525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:00.194993 23528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:00.194993 23526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:00.195124 23520 server_base.cc:1034] running on GCE node
I20260925 00:47:00.195605 23520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:00.196352 23520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:00.197532 23520 hybrid_clock.cc:648] HybridClock initialized: now 1790297220197524 us; error 43 us; skew 500 ppm
Sep 25 00:47:00 dist-test-slave-w8nh krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297220, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:00.200377 23520 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:47:00.201198 23520 webserver.cc:460] Webserver started at http://127.6.72.65:34581/ using document root <none> and password file <none>
I20260925 00:47:00.201763 23520 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:00.201853 23520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:00.202077 23520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:00.204064 23520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "cb8564eab27843fda001d6ee3b4825ab"
format_stamp: "Formatted at 2026-09-25 00:47:00 on dist-test-slave-w8nh"
server_key: "20df89499f8047deda73409582e881c2"
server_key_iv: "4a3497c24e5809919adae4a118046bda"
server_key_version: "encryptionkey@0"
I20260925 00:47:00.204679 23520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "cb8564eab27843fda001d6ee3b4825ab"
format_stamp: "Formatted at 2026-09-25 00:47:00 on dist-test-slave-w8nh"
server_key: "20df89499f8047deda73409582e881c2"
server_key_iv: "4a3497c24e5809919adae4a118046bda"
server_key_version: "encryptionkey@0"
I20260925 00:47:00.208454 23520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:47:00.211052 23535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:00.212376 23520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:47:00.212499 23520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "cb8564eab27843fda001d6ee3b4825ab"
format_stamp: "Formatted at 2026-09-25 00:47:00 on dist-test-slave-w8nh"
server_key: "20df89499f8047deda73409582e881c2"
server_key_iv: "4a3497c24e5809919adae4a118046bda"
server_key_version: "encryptionkey@0"
I20260925 00:47:00.212610 23520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:00.234907 23520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:00.238328 23520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:00.238535 23520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:00.239138 23520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:00.239984 23520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:00.240060 23520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:00.240132 23520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:00.240180 23520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:00.249940 23520 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:37441
I20260925 00:47:00.249962 23648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:37441 every 8 connection(s)
I20260925 00:47:00.250944 23520 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:47:00.256023 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 23520
I20260925 00:47:00.256174 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:47:00.256512 6433 external_mini_cluster.cc:1428] Setting key 0af5a363b5aa6df4f0596abfa8c2abe8
Sep 25 00:47:00 dist-test-slave-w8nh krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297220, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:00.265681 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.252869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:54099 (local address 127.6.72.126:44967)
0925 00:47:00.253025 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:00.253029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:00.253868 (+ 839us) server_negotiation.cc:407] Connection header received
0925 00:47:00.254809 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:00.254813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:00.254873 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:00.254974 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:00.256362 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.257065 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.258505 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.258753 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.258836 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.261458 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:00.261465 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:00.261471 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:00.261506 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:00.263359 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:00.263875 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:00.263881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:00.263883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:00.263986 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:00.264400 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:00.264404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:00.264406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:00.264666 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:00.264804 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:00.265324 (+ 520us) server_negotiation.cc:299] Negotiation successful
0925 00:47:00.265478 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 00:47:00.266454 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.253254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44967 (local address 127.6.72.65:54099)
0925 00:47:00.253716 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:00.253761 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:00.254572 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:00.255145 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:00.255154 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:00.255521 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:00.256192 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.256214 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.257187 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.257190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.258379 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.258393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.258904 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.258910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.259141 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.259779 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:00.259795 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:00.261287 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:00.263483 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:00.263488 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:00.263501 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:00.263769 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:00.264092 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:00.264094 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:00.264096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:00.264306 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:00.264793 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:00.264798 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:00.265042 (+ 244us) client_negotiation.cc:769] Sending connection context
0925 00:47:00.265208 (+ 166us) client_negotiation.cc:240] Negotiation successful
0925 00:47:00.265477 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:47:00.267931 23649 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44967
I20260925 00:47:00.268191 23649 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:00.268865 23649 heartbeater.cc:511] Master 127.6.72.126:44967 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:47:00.270547 23458 ts_manager.cc:194] Registered new tserver with Master: cb8564eab27843fda001d6ee3b4825ab (127.6.72.65:37441)
I20260925 00:47:00.271687 23458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:54099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:47:00.287287 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44967
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:40129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:00.397673 23654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:00.397944 23654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:00.397989 23654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:00.401938 23654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:00.401986 23654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:00.402043 23654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:47:00.407228 23654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:00.408188 23654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:00.409255 23654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:00.418675 23662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:00.418692 23659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:00.418784 23654 server_base.cc:1034] running on GCE node
W20260925 00:47:00.418694 23660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:00.419333 23654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:00.419979 23654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:00.421160 23654 hybrid_clock.cc:648] HybridClock initialized: now 1790297220421105 us; error 66 us; skew 500 ppm
Sep 25 00:47:00 dist-test-slave-w8nh krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297220, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:00.424141 23654 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:47:00.425146 23654 webserver.cc:460] Webserver started at http://127.6.72.66:44329/ using document root <none> and password file <none>
I20260925 00:47:00.425731 23654 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:00.425824 23654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:00.426057 23654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:00.428153 23654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "4907d19459e44e1a9fc14ee545f024bf"
format_stamp: "Formatted at 2026-09-25 00:47:00 on dist-test-slave-w8nh"
server_key: "65c3d8b77985dcacd6a601b45561ade7"
server_key_iv: "dcb52f577737c1943dd946d25086416b"
server_key_version: "encryptionkey@0"
I20260925 00:47:00.428808 23654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "4907d19459e44e1a9fc14ee545f024bf"
format_stamp: "Formatted at 2026-09-25 00:47:00 on dist-test-slave-w8nh"
server_key: "65c3d8b77985dcacd6a601b45561ade7"
server_key_iv: "dcb52f577737c1943dd946d25086416b"
server_key_version: "encryptionkey@0"
I20260925 00:47:00.432804 23654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:47:00.435369 23669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:00.436766 23654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:00.436908 23654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "4907d19459e44e1a9fc14ee545f024bf"
format_stamp: "Formatted at 2026-09-25 00:47:00 on dist-test-slave-w8nh"
server_key: "65c3d8b77985dcacd6a601b45561ade7"
server_key_iv: "dcb52f577737c1943dd946d25086416b"
server_key_version: "encryptionkey@0"
I20260925 00:47:00.437029 23654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:00.462175 23654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:00.465659 23654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:00.465865 23654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:00.466459 23654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:00.467398 23654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:00.467461 23654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:00.467525 23654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:00.467574 23654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:00.476485 23654 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:39033
I20260925 00:47:00.476512 23782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:39033 every 8 connection(s)
I20260925 00:47:00.477344 23654 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:47:00.484347 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 23654
I20260925 00:47:00.484478 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:47:00.484789 6433 external_mini_cluster.cc:1428] Setting key 4fe9f29d53aff686fc8c2b9e7f4b87cd
Sep 25 00:47:00 dist-test-slave-w8nh krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297220, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:00.490957 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.479219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:42497 (local address 127.6.72.126:44967)
0925 00:47:00.479385 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:00.479391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:00.480326 (+ 935us) server_negotiation.cc:407] Connection header received
0925 00:47:00.481208 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:00.481211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:00.481256 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:00.481329 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:00.482451 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.482962 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.484055 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.484441 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.484552 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.487167 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:00.487174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:00.487179 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:00.487213 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:00.488839 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:00.489440 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:00.489443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:00.489444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:00.489532 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:00.489907 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:00.489912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:00.489913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:00.490092 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:00.490198 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:00.490677 (+ 479us) server_negotiation.cc:299] Negotiation successful
0925 00:47:00.490795 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 00:47:00.491741 23785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.479603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44967 (local address 127.6.72.66:42497)
0925 00:47:00.480139 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:00.480181 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:00.481018 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:00.481490 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:00.481496 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:00.481852 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:00.482330 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.482343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.483062 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.483065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.483929 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.483937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.484589 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.484592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.484759 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.485411 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:00.485430 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:00.487005 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:00.488950 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:00.488959 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:00.488976 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:00.489338 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:00.489665 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:00.489669 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:00.489671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:00.489810 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:00.490197 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:00.490204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:00.490445 (+ 241us) client_negotiation.cc:769] Sending connection context
0925 00:47:00.490632 (+ 187us) client_negotiation.cc:240] Negotiation successful
0925 00:47:00.490943 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":110,"threads_started":1}
I20260925 00:47:00.492868 23783 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44967
I20260925 00:47:00.493113 23783 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:00.493579 23783 heartbeater.cc:511] Master 127.6.72.126:44967 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:47:00.494642 23458 ts_manager.cc:194] Registered new tserver with Master: 4907d19459e44e1a9fc14ee545f024bf (127.6.72.66:39033)
I20260925 00:47:00.495237 23458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:42497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:47:00.512074 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44967
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:40129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:00.622677 23788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:00.622939 23788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:00.622984 23788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:00.627077 23788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:00.627146 23788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:00.627211 23788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:47:00.632432 23788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:44967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:00.633491 23788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:00.634476 23788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:00.643975 23793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:00.644026 23794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:00.644026 23796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:00.644439 23788 server_base.cc:1034] running on GCE node
I20260925 00:47:00.644867 23788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:00.645519 23788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:00.646714 23788 hybrid_clock.cc:648] HybridClock initialized: now 1790297220646692 us; error 33 us; skew 500 ppm
Sep 25 00:47:00 dist-test-slave-w8nh krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297220, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:00.649744 23788 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:47:00.650688 23788 webserver.cc:460] Webserver started at http://127.6.72.67:40911/ using document root <none> and password file <none>
I20260925 00:47:00.651283 23788 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:00.651376 23788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:00.651611 23788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:00.653744 23788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "595264e58376465987b0f6f3e32dc6c3"
format_stamp: "Formatted at 2026-09-25 00:47:00 on dist-test-slave-w8nh"
server_key: "8a0296e4ad0b8807c42ff9d63689c1ee"
server_key_iv: "6f4a9821842c9792039b4203703e4b9c"
server_key_version: "encryptionkey@0"
I20260925 00:47:00.654322 23788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "595264e58376465987b0f6f3e32dc6c3"
format_stamp: "Formatted at 2026-09-25 00:47:00 on dist-test-slave-w8nh"
server_key: "8a0296e4ad0b8807c42ff9d63689c1ee"
server_key_iv: "6f4a9821842c9792039b4203703e4b9c"
server_key_version: "encryptionkey@0"
I20260925 00:47:00.658082 23788 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260925 00:47:00.660496 23803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:00.661700 23788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:47:00.661835 23788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "595264e58376465987b0f6f3e32dc6c3"
format_stamp: "Formatted at 2026-09-25 00:47:00 on dist-test-slave-w8nh"
server_key: "8a0296e4ad0b8807c42ff9d63689c1ee"
server_key_iv: "6f4a9821842c9792039b4203703e4b9c"
server_key_version: "encryptionkey@0"
I20260925 00:47:00.661945 23788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:00.679843 23788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:00.683183 23788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:00.683506 23788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:00.684129 23788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:00.685065 23788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:00.685135 23788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:00.685209 23788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:00.685245 23788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:00.694939 23788 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:39263
I20260925 00:47:00.694980 23916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:39263 every 8 connection(s)
I20260925 00:47:00.695881 23788 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:47:00.698802 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 23788
I20260925 00:47:00.698952 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:47:00.699299 6433 external_mini_cluster.cc:1428] Setting key a028bcce8721a22dee05d3fc1ca3ebc4
Sep 25 00:47:00 dist-test-slave-w8nh krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297220, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:00.709688 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.697883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:38759 (local address 127.6.72.126:44967)
0925 00:47:00.698024 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:00.698028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:00.698914 (+ 886us) server_negotiation.cc:407] Connection header received
0925 00:47:00.700161 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:00.700165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:00.700225 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:00.700341 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:00.701827 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.702358 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.703439 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.703647 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.703722 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.706149 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:00.706152 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:00.706154 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:00.706177 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:00.707647 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:00.708082 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:00.708085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:00.708086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:00.708157 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:00.708587 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:00.708591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:00.708592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:00.708799 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:00.708894 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:00.709401 (+ 507us) server_negotiation.cc:299] Negotiation successful
0925 00:47:00.709516 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 00:47:00.710461 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.698195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:44967 (local address 127.6.72.67:38759)
0925 00:47:00.698734 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:00.698801 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:00.699934 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:00.700547 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:00.700557 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:00.701066 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:00.701691 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.701713 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.702458 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.702461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.703338 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.703346 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.703783 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.703785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.703915 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.704507 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:00.704524 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:00.705977 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:00.707739 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:00.707744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:00.707754 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:00.707989 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:00.708287 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:00.708292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:00.708294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:00.708478 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:00.708907 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:00.708912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:00.709141 (+ 229us) client_negotiation.cc:769] Sending connection context
0925 00:47:00.709310 (+ 169us) client_negotiation.cc:240] Negotiation successful
0925 00:47:00.709563 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":110,"threads_started":1}
I20260925 00:47:00.711688 23917 heartbeater.cc:347] Connected to a master server at 127.6.72.126:44967
I20260925 00:47:00.711993 23917 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:00.712688 23917 heartbeater.cc:511] Master 127.6.72.126:44967 requested a full tablet report, sending...
I20260925 00:47:00.713937 23458 ts_manager.cc:194] Registered new tserver with Master: 595264e58376465987b0f6f3e32dc6c3 (127.6.72.67:39263)
I20260925 00:47:00.714533 23458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:38759
I20260925 00:47:00.725521 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:47:00.736181 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.727645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.6.72.126:44967)
0925 00:47:00.727776 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:00.727780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:00.727986 (+ 206us) server_negotiation.cc:407] Connection header received
0925 00:47:00.728152 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:00.728155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:00.728204 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:00.728403 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:00.729162 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.729738 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.730668 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.730901 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.730990 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.732564 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:00.732568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:00.732570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:00.732593 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:00.734277 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:00.734788 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:00.734791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:00.734792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:00.734875 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:00.735297 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:00.735300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:00.735301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:00.735530 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:00.735705 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:00.735959 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 00:47:00.736058 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 00:47:00.739264 23458 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:47:00.742329 23458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:47:00.756700 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.750291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:39263 (local address 127.0.0.1:58098)
0925 00:47:00.750756 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:00.750804 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:00.750929 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:00.752609 (+ 1680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:00.752612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:00.752625 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:00.752715 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.752719 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.753687 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.753690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.755381 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.755388 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.756135 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.756139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.756272 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.756312 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 00:47:00.756426 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 00:47:00.756545 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":276,"thread_start_us":97,"threads_started":1}
I20260925 00:47:00.756709 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.750460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58098 (local address 127.6.72.67:39263)
0925 00:47:00.751984 (+ 1524us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:00.751999 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:00.752023 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:47:00.752106 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:00.752128 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:00.752340 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:00.752476 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:00.752829 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.753548 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.755467 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.756009 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.756135 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.756320 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:00.756453 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 00:47:00.756546 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1257,"thread_start_us":102,"threads_started":1}
I20260925 00:47:00.757083 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.751188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48128 (local address 127.6.72.66:39033)
0925 00:47:00.751499 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:00.751510 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:00.751778 (+ 268us) server_negotiation.cc:407] Connection header received
0925 00:47:00.751858 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:00.751866 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:00.752035 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:00.752183 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:00.752454 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.753643 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.755262 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.756105 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.756212 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.756368 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:00.756834 (+ 466us) server_negotiation.cc:299] Negotiation successful
0925 00:47:00.756924 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":108,"threads_started":1}
I20260925 00:47:00.756956 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.750546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:37441 (local address 127.0.0.1:49762)
0925 00:47:00.751315 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:00.751330 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:00.751420 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:00.751951 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:00.751956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:00.751978 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:00.752142 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.752150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.753752 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.753755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.755258 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.755268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.756432 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.756436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.756633 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.756654 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 00:47:00.756728 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 00:47:00.756825 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":226,"thread_start_us":107,"threads_started":1}
I20260925 00:47:00.757151 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.751016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:39033 (local address 127.0.0.1:48128)
0925 00:47:00.751611 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:00.751622 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:00.751697 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:00.752168 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:00.752171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:00.752190 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:00.752329 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.752336 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.753816 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.753819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.755139 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.755148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.756432 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.756436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.756629 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.756669 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 00:47:00.756728 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 00:47:00.757034 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":419,"thread_start_us":93,"threads_started":1}
I20260925 00:47:00.757581 23932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.751001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.6.72.65:37441)
0925 00:47:00.751272 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:00.751287 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:00.751487 (+ 200us) server_negotiation.cc:407] Connection header received
0925 00:47:00.751577 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:00.751583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:00.751794 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:00.752302 (+ 508us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:00.752466 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.753642 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.755387 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.756126 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.756213 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.756368 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:00.757313 (+ 945us) server_negotiation.cc:299] Negotiation successful
0925 00:47:00.757412 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":103,"threads_started":1}
I20260925 00:47:00.759987 23851 tablet_service.cc:1467] Processing CreateTablet for tablet 6270c8099c3443baa83708870b72f983 (DEFAULT_TABLE table=test-table [id=438b612514a044999c54d455fba5e5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:00.760061 23717 tablet_service.cc:1467] Processing CreateTablet for tablet 6270c8099c3443baa83708870b72f983 (DEFAULT_TABLE table=test-table [id=438b612514a044999c54d455fba5e5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:00.760519 23583 tablet_service.cc:1467] Processing CreateTablet for tablet 6270c8099c3443baa83708870b72f983 (DEFAULT_TABLE table=test-table [id=438b612514a044999c54d455fba5e5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:00.761106 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6270c8099c3443baa83708870b72f983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:00.761130 23717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6270c8099c3443baa83708870b72f983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:00.761396 23583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6270c8099c3443baa83708870b72f983. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:00.768383 23935 tablet_bootstrap.cc:492] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab: Bootstrap starting.
I20260925 00:47:00.768939 23936 tablet_bootstrap.cc:492] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: Bootstrap starting.
I20260925 00:47:00.769163 23934 tablet_bootstrap.cc:492] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: Bootstrap starting.
I20260925 00:47:00.770438 23935 tablet_bootstrap.cc:654] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:00.771096 23934 tablet_bootstrap.cc:654] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:00.771214 23935 log.cc:824] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:00.771538 23936 tablet_bootstrap.cc:654] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:00.772357 23936 log.cc:824] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:00.772357 23934 log.cc:824] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:00.773420 23935 tablet_bootstrap.cc:492] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab: No bootstrap required, opened a new log
I20260925 00:47:00.773651 23935 ts_tablet_manager.cc:1397] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260925 00:47:00.774322 23934 tablet_bootstrap.cc:492] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: No bootstrap required, opened a new log
I20260925 00:47:00.774499 23934 ts_tablet_manager.cc:1397] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260925 00:47:00.774653 23936 tablet_bootstrap.cc:492] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: No bootstrap required, opened a new log
I20260925 00:47:00.774822 23936 ts_tablet_manager.cc:1397] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:47:00.777595 23936 raft_consensus.cc:348] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 } } peers { permanent_uuid: "cb8564eab27843fda001d6ee3b4825ab" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 37441 } } peers { permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 } }
I20260925 00:47:00.777798 23936 raft_consensus.cc:374] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:00.777835 23936 raft_consensus.cc:729] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4907d19459e44e1a9fc14ee545f024bf, State: Initialized, Role: FOLLOWER
I20260925 00:47:00.778264 23936 consensus_queue.cc:260] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 } } peers { permanent_uuid: "cb8564eab27843fda001d6ee3b4825ab" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 37441 } } peers { permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 } }
I20260925 00:47:00.778182 23935 raft_consensus.cc:348] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 } } peers { permanent_uuid: "cb8564eab27843fda001d6ee3b4825ab" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 37441 } } peers { permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 } }
I20260925 00:47:00.778182 23934 raft_consensus.cc:348] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 } } peers { permanent_uuid: "cb8564eab27843fda001d6ee3b4825ab" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 37441 } } peers { permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 } }
I20260925 00:47:00.778422 23934 raft_consensus.cc:374] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:00.778422 23935 raft_consensus.cc:374] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:00.778481 23935 raft_consensus.cc:729] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb8564eab27843fda001d6ee3b4825ab, State: Initialized, Role: FOLLOWER
I20260925 00:47:00.778512 23934 raft_consensus.cc:729] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 595264e58376465987b0f6f3e32dc6c3, State: Initialized, Role: FOLLOWER
I20260925 00:47:00.778910 23783 heartbeater.cc:503] Master 127.6.72.126:44967 was elected leader, sending a full tablet report...
I20260925 00:47:00.778942 23935 consensus_queue.cc:260] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 } } peers { permanent_uuid: "cb8564eab27843fda001d6ee3b4825ab" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 37441 } } peers { permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 } }
I20260925 00:47:00.778928 23934 consensus_queue.cc:260] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 } } peers { permanent_uuid: "cb8564eab27843fda001d6ee3b4825ab" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 37441 } } peers { permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 } }
I20260925 00:47:00.779235 23936 ts_tablet_manager.cc:1428] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260925 00:47:00.779835 23934 ts_tablet_manager.cc:1428] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 00:47:00.780200 23917 heartbeater.cc:503] Master 127.6.72.126:44967 was elected leader, sending a full tablet report...
I20260925 00:47:00.780339 23935 ts_tablet_manager.cc:1428] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260925 00:47:00.780663 23649 heartbeater.cc:503] Master 127.6.72.126:44967 was elected leader, sending a full tablet report...
I20260925 00:47:00.930599 23942 raft_consensus.cc:482] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:47:00.930814 23942 raft_consensus.cc:504] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 } } peers { permanent_uuid: "cb8564eab27843fda001d6ee3b4825ab" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 37441 } } peers { permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 } }
I20260925 00:47:00.931994 23942 leader_election.cc:288] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 595264e58376465987b0f6f3e32dc6c3 (127.6.72.67:39263), 4907d19459e44e1a9fc14ee545f024bf (127.6.72.66:39033)
I20260925 00:47:00.936463 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.932455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:52147 (local address 127.6.72.67:39263)
0925 00:47:00.932653 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:00.932658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:00.932865 (+ 207us) server_negotiation.cc:407] Connection header received
0925 00:47:00.933053 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:00.933057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:00.933112 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:00.933206 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:00.933583 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.934166 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.935355 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.935810 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.935894 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.935946 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:00.936214 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 00:47:00.936331 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 00:47:00.936479 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.932221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:39263 (local address 127.6.72.65:52147)
0925 00:47:00.932755 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:00.932771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:00.932915 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:00.933259 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:00.933266 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:00.933293 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:00.933461 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.933471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.934285 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.934288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.935226 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.935236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.935959 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.935963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.936060 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.936099 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 00:47:00.936187 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 00:47:00.936319 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":1,"thread_start_us":134,"threads_started":1}
I20260925 00:47:00.936731 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.932785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:39033 (local address 127.6.72.65:35977)
0925 00:47:00.933129 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:00.933145 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:00.933253 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:00.933542 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:00.933548 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:00.933568 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:00.933679 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.933685 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.934478 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.934481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.935367 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:00.935377 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.936155 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.936178 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:00.936370 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.936394 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 00:47:00.936478 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 00:47:00.936558 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":41,"thread_start_us":132,"threads_started":1}
I20260925 00:47:00.936774 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.932512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:35977 (local address 127.6.72.66:39033)
0925 00:47:00.932669 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:00.932675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:00.933238 (+ 563us) server_negotiation.cc:407] Connection header received
0925 00:47:00.933359 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:00.933362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:00.933417 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:00.933513 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:00.933793 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.934356 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.935521 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.936032 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.936149 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.936218 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:00.936515 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 00:47:00.936597 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:47:00.937321 23871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6270c8099c3443baa83708870b72f983" candidate_uuid: "cb8564eab27843fda001d6ee3b4825ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "595264e58376465987b0f6f3e32dc6c3" is_pre_election: true
I20260925 00:47:00.937521 23737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6270c8099c3443baa83708870b72f983" candidate_uuid: "cb8564eab27843fda001d6ee3b4825ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4907d19459e44e1a9fc14ee545f024bf" is_pre_election: true
I20260925 00:47:00.937706 23871 raft_consensus.cc:2454] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb8564eab27843fda001d6ee3b4825ab in term 0.
I20260925 00:47:00.937834 23737 raft_consensus.cc:2454] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb8564eab27843fda001d6ee3b4825ab in term 0.
I20260925 00:47:00.938277 23539 leader_election.cc:302] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595264e58376465987b0f6f3e32dc6c3, cb8564eab27843fda001d6ee3b4825ab; no voters:
I20260925 00:47:00.938560 23942 raft_consensus.cc:2781] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:47:00.938648 23942 raft_consensus.cc:482] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:47:00.938692 23942 raft_consensus.cc:3037] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:00.939759 23942 raft_consensus.cc:504] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 } } peers { permanent_uuid: "cb8564eab27843fda001d6ee3b4825ab" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 37441 } } peers { permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 } }
I20260925 00:47:00.940171 23942 leader_election.cc:288] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [CANDIDATE]: Term 1 election: Requested vote from peers 595264e58376465987b0f6f3e32dc6c3 (127.6.72.67:39263), 4907d19459e44e1a9fc14ee545f024bf (127.6.72.66:39033)
I20260925 00:47:00.940543 23871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6270c8099c3443baa83708870b72f983" candidate_uuid: "cb8564eab27843fda001d6ee3b4825ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "595264e58376465987b0f6f3e32dc6c3"
I20260925 00:47:00.940605 23737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6270c8099c3443baa83708870b72f983" candidate_uuid: "cb8564eab27843fda001d6ee3b4825ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4907d19459e44e1a9fc14ee545f024bf"
I20260925 00:47:00.940680 23871 raft_consensus.cc:3037] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:00.940768 23737 raft_consensus.cc:3037] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:00.942024 23871 raft_consensus.cc:2454] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb8564eab27843fda001d6ee3b4825ab in term 1.
I20260925 00:47:00.942024 23737 raft_consensus.cc:2454] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb8564eab27843fda001d6ee3b4825ab in term 1.
I20260925 00:47:00.942332 23539 leader_election.cc:302] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595264e58376465987b0f6f3e32dc6c3, cb8564eab27843fda001d6ee3b4825ab; no voters:
I20260925 00:47:00.942504 23942 raft_consensus.cc:2781] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:00.942739 23942 raft_consensus.cc:686] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 1 LEADER]: Becoming Leader. State: Replica: cb8564eab27843fda001d6ee3b4825ab, State: Running, Role: LEADER
I20260925 00:47:00.943109 23942 consensus_queue.cc:237] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 } } peers { permanent_uuid: "cb8564eab27843fda001d6ee3b4825ab" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 37441 } } peers { permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 } }
I20260925 00:47:00.946533 23454 catalog_manager.cc:5495] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab reported cstate change: term changed from 0 to 1, leader changed from <none> to cb8564eab27843fda001d6ee3b4825ab (127.6.72.65). New cstate: current_term: 1 leader_uuid: "cb8564eab27843fda001d6ee3b4825ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb8564eab27843fda001d6ee3b4825ab" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 37441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 } health_report { overall_health: UNKNOWN } } }
W20260925 00:47:00.946887 23918 tablet.cc:2369] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:47:00.978549 23784 tablet.cc:2369] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:47:00.991907 23932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:00.988487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49778 (local address 127.6.72.65:37441)
0925 00:47:00.988620 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:00.988624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:00.988640 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:47:00.988812 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:00.988815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:00.988872 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:00.988962 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:00.989188 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.989678 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.990568 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:00.990775 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:00.990849 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:00.991108 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:00.991581 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:00.991698 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:00.991751 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 00:47:00.991797 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 00:47:00.997653 23737 raft_consensus.cc:1261] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf [term 1 FOLLOWER]: Refusing update from remote peer cb8564eab27843fda001d6ee3b4825ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:00.997654 23871 raft_consensus.cc:1261] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3 [term 1 FOLLOWER]: Refusing update from remote peer cb8564eab27843fda001d6ee3b4825ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:00.998356 23945 consensus_queue.cc:1035] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "4907d19459e44e1a9fc14ee545f024bf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:00.998581 23942 consensus_queue.cc:1035] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "595264e58376465987b0f6f3e32dc6c3" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 00:47:01.002277 23650 tablet.cc:2369] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:47:01.006342 23948 mvcc.cc:205] Tried to move back new op lower bound from 7333057417199316992 to 7333057416985608192. Current Snapshot: MvccSnapshot[applied={T|T < 7333057417199316992}]
I20260925 00:47:01.009966 23949 mvcc.cc:205] Tried to move back new op lower bound from 7333057417199316992 to 7333057416985608192. Current Snapshot: MvccSnapshot[applied={T|T < 7333057417199316992 or (T in {7333057417199316992})}]
I20260925 00:47:01.015688 23458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:01.015899 23458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:01.018587 23458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c30b77638a447439db274976733974a: Sending DeleteTablet for 3 replicas of tablet 6270c8099c3443baa83708870b72f983
I20260925 00:47:01.019425 23583 tablet_service.cc:1514] Processing DeleteTablet for tablet 6270c8099c3443baa83708870b72f983 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:01 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:49762
I20260925 00:47:01.019407 23851 tablet_service.cc:1514] Processing DeleteTablet for tablet 6270c8099c3443baa83708870b72f983 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:01 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:58098
I20260925 00:47:01.019943 23958 tablet_replica.cc:330] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab: stopping tablet replica
I20260925 00:47:01.019963 23959 tablet_replica.cc:330] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: stopping tablet replica
I20260925 00:47:01.019963 23717 tablet_service.cc:1514] Processing DeleteTablet for tablet 6270c8099c3443baa83708870b72f983 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:01 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:48128
I20260925 00:47:01.020320 23958 raft_consensus.cc:2229] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:47:01.020320 23959 raft_consensus.cc:2229] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:01.020653 23959 raft_consensus.cc:2258] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:01.020689 23958 raft_consensus.cc:2258] T 6270c8099c3443baa83708870b72f983 P cb8564eab27843fda001d6ee3b4825ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:01.020980 23960 tablet_replica.cc:330] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: stopping tablet replica
I20260925 00:47:01.021340 23960 raft_consensus.cc:2229] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:01.021400 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 23520
I20260925 00:47:01.022892 23960 raft_consensus.cc:2258] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:01.022941 23959 ts_tablet_manager.cc:1905] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:01.024562 23960 ts_tablet_manager.cc:1905] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:01.025619 23959 ts_tablet_manager.cc:1918] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:01.025781 23959 log.cc:1196] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/6270c8099c3443baa83708870b72f983
I20260925 00:47:01.026127 23959 ts_tablet_manager.cc:1939] T 6270c8099c3443baa83708870b72f983 P 595264e58376465987b0f6f3e32dc6c3: Deleting consensus metadata
I20260925 00:47:01.027297 23445 catalog_manager.cc:4841] TS 595264e58376465987b0f6f3e32dc6c3 (127.6.72.67:39263): tablet 6270c8099c3443baa83708870b72f983 (table test-table [id=438b612514a044999c54d455fba5e5cf]) successfully deleted
I20260925 00:47:01.027858 23960 ts_tablet_manager.cc:1918] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:01.027961 23960 log.cc:1196] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/6270c8099c3443baa83708870b72f983
I20260925 00:47:01.028304 23960 ts_tablet_manager.cc:1939] T 6270c8099c3443baa83708870b72f983 P 4907d19459e44e1a9fc14ee545f024bf: Deleting consensus metadata
W20260925 00:47:01.029692 23444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:37441 (error 108)
I20260925 00:47:01.029946 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 23654
I20260925 00:47:01.030228 23444 catalog_manager.cc:4841] TS 4907d19459e44e1a9fc14ee545f024bf (127.6.72.66:39033): tablet 6270c8099c3443baa83708870b72f983 (table test-table [id=438b612514a044999c54d455fba5e5cf]) successfully deleted
I20260925 00:47:01.030952 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.030522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:37441 (local address 127.0.0.1:49784)
0925 00:47:01.030774 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:01.030851 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:37441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260925 00:47:01.031111 23444 catalog_manager.cc:4584] TS cb8564eab27843fda001d6ee3b4825ab (127.6.72.65:37441): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6270c8099c3443baa83708870b72f983: Network error: Client connection negotiation failed: client connection to 127.6.72.65:37441: connect: Connection refused (error 111)
I20260925 00:47:01.035833 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 23788
I20260925 00:47:01.041491 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 23426
2026-09-25T00:47:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2476 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T00:47:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:01Z Disabled control of system clock
I20260925 00:47:01.072592 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41691
--webserver_interface=127.6.72.126
--webserver_port=0
--builtin_ntp_servers=127.6.72.84:40737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:41691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 00:47:01.187595 23965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:01.187891 23965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:01.187929 23965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:01.192186 23965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:01.192305 23965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:01.192322 23965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:01.192340 23965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:01.192356 23965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:01.197278 23965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:41691
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:41691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:01.198501 23965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:01.199973 23965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:01.209156 23973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:01.209169 23971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:01.209272 23965 server_base.cc:1034] running on GCE node
W20260925 00:47:01.209165 23970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:01.210366 23965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:01.211476 23965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:01.212707 23965 hybrid_clock.cc:648] HybridClock initialized: now 1790297221212679 us; error 40 us; skew 500 ppm
I20260925 00:47:01.214591 23965 webserver.cc:460] Webserver started at http://127.6.72.126:37265/ using document root <none> and password file <none>
I20260925 00:47:01.215190 23965 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:01.215287 23965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:01.215523 23965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:01.217723 23965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "534eb2657b144da79f9d36a355ff58d8"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "d5890e2bc36c6e122f04cf352665e4f1"
server_key_iv: "1e6d106feb405ed2381b56fd97cea28a"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.218323 23965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "534eb2657b144da79f9d36a355ff58d8"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "d5890e2bc36c6e122f04cf352665e4f1"
server_key_iv: "1e6d106feb405ed2381b56fd97cea28a"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.222263 23965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:47:01.224797 23979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:01.226042 23965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:47:01.226198 23965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "534eb2657b144da79f9d36a355ff58d8"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "d5890e2bc36c6e122f04cf352665e4f1"
server_key_iv: "1e6d106feb405ed2381b56fd97cea28a"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.226320 23965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:01.252460 23965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:01.253307 23965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:01.253532 23965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:01.261023 23965 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:41691
I20260925 00:47:01.261029 24031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:41691 every 8 connection(s)
I20260925 00:47:01.262080 23965 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:01.265368 24032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:01.268908 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 23965
I20260925 00:47:01.269089 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:01.269402 6433 external_mini_cluster.cc:1428] Setting key ffa32401e9464438052ee51f0c4fcedb
I20260925 00:47:01.271452 24032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8: Bootstrap starting.
I20260925 00:47:01.273841 24032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:01.274605 24032 log.cc:824] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:01.276613 24032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8: No bootstrap required, opened a new log
I20260925 00:47:01.279546 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.271073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55010 (local address 127.6.72.126:41691)
0925 00:47:01.271837 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:01.271853 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:01.271891 (+ 38us) server_negotiation.cc:407] Connection header received
0925 00:47:01.272663 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:01.272683 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:01.273104 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:01.273453 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:01.273903 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.274990 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.275903 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.276439 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.276609 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.277641 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:01.277647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:01.277662 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:01.278104 (+ 442us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:01.278165 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:01.278187 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:01.278313 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:01.278478 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 00:47:01.278765 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":155,"threads_started":1}
I20260925 00:47:01.280521 24032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534eb2657b144da79f9d36a355ff58d8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41691 } }
I20260925 00:47:01.280784 24032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:01.280872 24032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 534eb2657b144da79f9d36a355ff58d8, State: Initialized, Role: FOLLOWER
I20260925 00:47:01.281311 24032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534eb2657b144da79f9d36a355ff58d8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41691 } }
I20260925 00:47:01.281462 24032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:01.281546 24032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:01.281661 24032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:01.282703 24032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534eb2657b144da79f9d36a355ff58d8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41691 } }
I20260925 00:47:01.283053 24032 leader_election.cc:302] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 534eb2657b144da79f9d36a355ff58d8; no voters:
I20260925 00:47:01.283411 24032 leader_election.cc:288] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:01.283565 24037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:01.283806 24037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [term 1 LEADER]: Becoming Leader. State: Replica: 534eb2657b144da79f9d36a355ff58d8, State: Running, Role: LEADER
I20260925 00:47:01.284148 24037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534eb2657b144da79f9d36a355ff58d8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41691 } }
I20260925 00:47:01.284739 24032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:01.285877 24039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "534eb2657b144da79f9d36a355ff58d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534eb2657b144da79f9d36a355ff58d8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41691 } } }
I20260925 00:47:01.285877 24038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 534eb2657b144da79f9d36a355ff58d8. Latest consensus state: current_term: 1 leader_uuid: "534eb2657b144da79f9d36a355ff58d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "534eb2657b144da79f9d36a355ff58d8" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 41691 } } }
I20260925 00:47:01.286032 24039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:01.286032 24038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:01.286440 24046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:01.289419 24046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:01.295499 24046 catalog_manager.cc:1347] Generated new cluster ID: 7aeee4a58b244584bffc0b3b63fdc42e
I20260925 00:47:01.295606 24046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:01.307700 24046 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:01.308983 24046 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:01.322155 24046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8: Generated new TSK 0
I20260925 00:47:01.322984 24046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:01.330612 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41691
--builtin_ntp_servers=127.6.72.84:40737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 00:47:01.445359 24056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:01.445644 24056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:01.445684 24056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:01.449839 24056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:01.449913 24056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:01.450017 24056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:47:01.455070 24056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:01.456305 24056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:01.457762 24056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:01.467588 24061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:01.467635 24064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:01.467593 24062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:01.468056 24056 server_base.cc:1034] running on GCE node
I20260925 00:47:01.468523 24056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:01.469233 24056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:01.470417 24056 hybrid_clock.cc:648] HybridClock initialized: now 1790297221470387 us; error 45 us; skew 500 ppm
I20260925 00:47:01.472476 24056 webserver.cc:460] Webserver started at http://127.6.72.65:44193/ using document root <none> and password file <none>
I20260925 00:47:01.473073 24056 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:01.473163 24056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:01.473392 24056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:01.475528 24056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "1bccb3ea5fbd412d9c28b856d20d3f11"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "ca4e324b87ed6af290f30158c2382c0c"
server_key_iv: "87e6888c95e0125dd7c9db9b58be5198"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.476128 24056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "1bccb3ea5fbd412d9c28b856d20d3f11"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "ca4e324b87ed6af290f30158c2382c0c"
server_key_iv: "87e6888c95e0125dd7c9db9b58be5198"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.479971 24056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260925 00:47:01.482614 24070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:01.483942 24056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 00:47:01.484086 24056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "1bccb3ea5fbd412d9c28b856d20d3f11"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "ca4e324b87ed6af290f30158c2382c0c"
server_key_iv: "87e6888c95e0125dd7c9db9b58be5198"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.484208 24056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:01.500977 24056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:01.501727 24056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:01.501937 24056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:01.502556 24056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:01.503549 24056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:01.503623 24056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:01.503695 24056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:01.503746 24056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:01.513073 24056 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:39953
I20260925 00:47:01.513159 24183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:39953 every 8 connection(s)
I20260925 00:47:01.514097 24056 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:47:01.517539 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 24056
I20260925 00:47:01.517688 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:47:01.517982 6433 external_mini_cluster.cc:1428] Setting key e0641861adc740d8bad92b72e8120626
I20260925 00:47:01.520192 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41691
--builtin_ntp_servers=127.6.72.84:40737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 00:47:01.525046 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.516123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:43157 (local address 127.6.72.126:41691)
0925 00:47:01.516321 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:01.516324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:01.517441 (+ 1117us) server_negotiation.cc:407] Connection header received
0925 00:47:01.519060 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:01.519062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:01.519106 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:01.519176 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:01.520454 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.521340 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.522739 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.522982 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.523105 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.524485 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:01.524488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:01.524490 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:01.524507 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:01.524530 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:01.524538 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:01.524630 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:01.524845 (+ 215us) server_negotiation.cc:299] Negotiation successful
0925 00:47:01.524906 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:47:01.526450 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.516547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41691 (local address 127.6.72.65:43157)
0925 00:47:01.517259 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:01.517301 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:01.518832 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:01.519385 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:01.519393 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:01.519509 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:01.520218 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.520289 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.521518 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.521524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.522587 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.522597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.523156 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.523160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.523342 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.524086 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:47:01.524090 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:01.524158 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:47:01.524296 (+ 138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:01.524649 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:01.524663 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:01.524672 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 00:47:01.524829 (+ 157us) client_negotiation.cc:240] Negotiation successful
0925 00:47:01.525206 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":114,"threads_started":1}
I20260925 00:47:01.527889 24184 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41691
I20260925 00:47:01.528184 24184 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:01.529002 24184 heartbeater.cc:511] Master 127.6.72.126:41691 requested a full tablet report, sending...
I20260925 00:47:01.531035 23996 ts_manager.cc:194] Registered new tserver with Master: 1bccb3ea5fbd412d9c28b856d20d3f11 (127.6.72.65:39953)
I20260925 00:47:01.532886 23996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.65:43157
W20260925 00:47:01.637137 24187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:01.637429 24187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:01.637470 24187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:01.641466 24187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:01.641523 24187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:01.641579 24187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:47:01.646267 24187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:01.647395 24187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:01.648777 24187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:01.658563 24192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:01.658571 24193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:01.658602 24195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:01.658625 24187 server_base.cc:1034] running on GCE node
I20260925 00:47:01.659251 24187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:01.659894 24187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:01.661055 24187 hybrid_clock.cc:648] HybridClock initialized: now 1790297221661005 us; error 64 us; skew 500 ppm
I20260925 00:47:01.663023 24187 webserver.cc:460] Webserver started at http://127.6.72.66:37385/ using document root <none> and password file <none>
I20260925 00:47:01.663750 24187 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:01.663841 24187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:01.664088 24187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:01.666280 24187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "b4333820f3f2433abc6cfda5239e18fd"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "ddef3813ba5b26739ba45d04d4f4dc29"
server_key_iv: "d54ee3981d4e660b6beafef18d0c9972"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.666813 24187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "b4333820f3f2433abc6cfda5239e18fd"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "ddef3813ba5b26739ba45d04d4f4dc29"
server_key_iv: "d54ee3981d4e660b6beafef18d0c9972"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.670641 24187 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:47:01.673064 24201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:01.674358 24187 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:47:01.674492 24187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "b4333820f3f2433abc6cfda5239e18fd"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "ddef3813ba5b26739ba45d04d4f4dc29"
server_key_iv: "d54ee3981d4e660b6beafef18d0c9972"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.674608 24187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:01.699529 24187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:01.700361 24187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:01.700567 24187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:01.701166 24187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:01.702138 24187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:01.702219 24187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:01.702292 24187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:01.702342 24187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:01.711473 24187 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:39507
I20260925 00:47:01.711496 24314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:39507 every 8 connection(s)
I20260925 00:47:01.712442 24187 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:47:01.717509 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 24187
I20260925 00:47:01.717685 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:47:01.717983 6433 external_mini_cluster.cc:1428] Setting key f7c5123990710c59b18e772efedef603
I20260925 00:47:01.719969 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41691
--builtin_ntp_servers=127.6.72.84:40737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 00:47:01.722091 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.714305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:35341 (local address 127.6.72.126:41691)
0925 00:47:01.714452 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:01.714456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:01.715485 (+ 1029us) server_negotiation.cc:407] Connection header received
0925 00:47:01.716983 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:01.716986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:01.717046 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:01.717154 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:01.718178 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.718795 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.719974 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.720309 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.720435 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.721590 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:01.721594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:01.721597 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:01.721611 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:01.721637 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:01.721646 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:01.721726 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:01.721890 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 00:47:01.721952 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:47:01.723080 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.714688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41691 (local address 127.6.72.66:35341)
0925 00:47:01.715332 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:01.715374 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:01.716770 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:01.717333 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:01.717340 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:01.717427 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:01.718005 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.718019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.718943 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.718946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.719843 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.719852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.720475 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.720478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.720650 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.721299 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:47:01.721300 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:01.721343 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:47:01.721409 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:01.721727 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:01.721739 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:01.721747 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 00:47:01.721878 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 00:47:01.722181 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":113,"threads_started":1}
I20260925 00:47:01.725298 24315 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41691
I20260925 00:47:01.725631 24315 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:01.726351 24315 heartbeater.cc:511] Master 127.6.72.126:41691 requested a full tablet report, sending...
I20260925 00:47:01.729296 23996 ts_manager.cc:194] Registered new tserver with Master: b4333820f3f2433abc6cfda5239e18fd (127.6.72.66:39507)
I20260925 00:47:01.730481 23996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.66:35341
W20260925 00:47:01.851187 24318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:01.851467 24318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:01.851506 24318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:01.855615 24318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:01.855729 24318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:01.855829 24318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:47:01.860721 24318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:41691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:01.861896 24318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:01.863368 24318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:01.873716 24326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:01.873703 24324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:01.873677 24323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:01.874011 24318 server_base.cc:1034] running on GCE node
I20260925 00:47:01.874553 24318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:01.875376 24318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:01.876566 24318 hybrid_clock.cc:648] HybridClock initialized: now 1790297221876527 us; error 60 us; skew 500 ppm
I20260925 00:47:01.878567 24318 webserver.cc:460] Webserver started at http://127.6.72.67:41153/ using document root <none> and password file <none>
I20260925 00:47:01.879197 24318 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:01.879292 24318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:01.879508 24318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:01.881705 24318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "301ec7153f5c4fa19d225e93a7eacdd4"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "4b988966845ec63567b2a2399c8645cd"
server_key_iv: "cf7c3d64c87be1d1b18f06b41550e29d"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.882304 24318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "301ec7153f5c4fa19d225e93a7eacdd4"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "4b988966845ec63567b2a2399c8645cd"
server_key_iv: "cf7c3d64c87be1d1b18f06b41550e29d"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.886477 24318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:47:01.889300 24332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:01.890632 24318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 00:47:01.890781 24318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "301ec7153f5c4fa19d225e93a7eacdd4"
format_stamp: "Formatted at 2026-09-25 00:47:01 on dist-test-slave-w8nh"
server_key: "4b988966845ec63567b2a2399c8645cd"
server_key_iv: "cf7c3d64c87be1d1b18f06b41550e29d"
server_key_version: "encryptionkey@0"
I20260925 00:47:01.890906 24318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:01.910897 24318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:01.911638 24318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:01.911856 24318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:01.912537 24318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:01.913530 24318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:01.913604 24318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:01.913677 24318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:01.913728 24318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:01.922833 24318 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:44147
I20260925 00:47:01.922863 24445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:44147 every 8 connection(s)
I20260925 00:47:01.923804 24318 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:47:01.926909 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 24318
I20260925 00:47:01.927040 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:47:01.927395 6433 external_mini_cluster.cc:1428] Setting key 61b2a34cae74ec1f4d988813b6ac6fe7
I20260925 00:47:01.934011 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.925769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:40649 (local address 127.6.72.126:41691)
0925 00:47:01.925907 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:01.925911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:01.927104 (+ 1193us) server_negotiation.cc:407] Connection header received
0925 00:47:01.929002 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:01.929005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:01.929052 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:01.929151 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:01.930364 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.930940 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.932127 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.932392 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.932475 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.933608 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:01.933611 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:01.933613 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:01.933625 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:01.933640 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:01.933648 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:01.933701 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:01.933846 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 00:47:01.933895 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:47:01.934929 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.926126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:41691 (local address 127.6.72.67:40649)
0925 00:47:01.926886 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:01.926945 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:01.928709 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:01.929492 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:01.929499 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:01.929587 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:01.930229 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.930243 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.931092 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.931096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.932018 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.932027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.932527 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.932530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.932707 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.933372 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:47:01.933373 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:01.933414 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:47:01.933480 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:01.933719 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:01.933725 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:01.933729 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 00:47:01.933834 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 00:47:01.934089 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260925 00:47:01.936045 24446 heartbeater.cc:347] Connected to a master server at 127.6.72.126:41691
I20260925 00:47:01.936334 24446 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:01.936865 24446 heartbeater.cc:511] Master 127.6.72.126:41691 requested a full tablet report, sending...
I20260925 00:47:01.937887 23996 ts_manager.cc:194] Registered new tserver with Master: 301ec7153f5c4fa19d225e93a7eacdd4 (127.6.72.67:44147)
I20260925 00:47:01.938470 23996 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.67:40649
I20260925 00:47:01.941524 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:47:01.946959 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.943498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55026 (local address 127.6.72.126:41691)
0925 00:47:01.943647 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:01.943654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:01.943808 (+ 154us) server_negotiation.cc:407] Connection header received
0925 00:47:01.943898 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:01.943900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:01.943940 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:01.944004 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:01.944277 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.944761 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.945563 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.945757 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.945830 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.946550 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:01.946553 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:01.946555 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:01.946567 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:01.946584 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:01.946591 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:01.946647 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:01.946752 (+ 105us) server_negotiation.cc:299] Negotiation successful
0925 00:47:01.946797 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 00:47:01.949692 23996 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:47:01.952711 23996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:47:01.967258 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.961583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.6.72.66:39507)
0925 00:47:01.962652 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:01.962665 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:01.962689 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:47:01.962773 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:01.962776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:01.962935 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:01.963103 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:01.963373 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.964497 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.965995 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.966572 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.966698 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.966909 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:01.966993 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 00:47:01.967087 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":175,"threads_started":1}
I20260925 00:47:01.967257 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.961444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:39507 (local address 127.0.0.1:43918)
0925 00:47:01.961937 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:01.961986 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:01.962140 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:01.963092 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:01.963098 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:01.963121 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:01.963270 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.963279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.964647 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.964649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.965869 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.965878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.966698 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.966702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.966809 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.966844 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 00:47:01.966963 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 00:47:01.967087 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":37,"thread_start_us":98,"threads_started":1}
I20260925 00:47:01.967585 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.962189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:39953 (local address 127.0.0.1:44706)
0925 00:47:01.962402 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:01.962416 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:01.962506 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:01.964014 (+ 1508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:01.964017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:01.964034 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:01.964108 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.964112 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.965412 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.965415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.966164 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.966173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.967193 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.967197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.967290 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.967325 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 00:47:01.967370 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 00:47:01.967435 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":12,"thread_start_us":89,"threads_started":1}
I20260925 00:47:01.968359 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.963160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.6.72.65:39953)
0925 00:47:01.963559 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:01.963570 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:01.963591 (+ 21us) server_negotiation.cc:407] Connection header received
0925 00:47:01.963663 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:01.963670 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:01.963872 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:01.964019 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:01.964224 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.965296 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.966289 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.967064 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.967852 (+ 788us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.968024 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:01.968073 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 00:47:01.968159 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":101,"threads_started":1}
I20260925 00:47:01.968876 24457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.961532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:44147 (local address 127.0.0.1:60934)
0925 00:47:01.962165 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:01.962178 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:01.962272 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:01.963584 (+ 1312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:01.963587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:01.963605 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:01.963704 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.963710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.965177 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.965180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.967230 (+ 2050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:01.967238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.968504 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.968508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:01.968600 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.968621 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 00:47:01.968695 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 00:47:01.968758 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":27,"thread_start_us":117,"threads_started":1}
I20260925 00:47:01.969327 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:01.962775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.6.72.67:44147)
0925 00:47:01.963092 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:01.963104 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:01.963128 (+ 24us) server_negotiation.cc:407] Connection header received
0925 00:47:01.963201 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:01.963206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:01.963405 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:01.963558 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:01.963822 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.965029 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.967355 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:01.968136 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:01.968236 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:01.968436 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:01.969042 (+ 606us) server_negotiation.cc:299] Negotiation successful
0925 00:47:01.969149 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":137,"threads_started":1}
I20260925 00:47:01.970866 24249 tablet_service.cc:1467] Processing CreateTablet for tablet 57c1fca980d1415396eb94c3aadd77dd (DEFAULT_TABLE table=test-table [id=6fc3d7714c5e48b0a6ee4b829432debd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:01.971045 24118 tablet_service.cc:1467] Processing CreateTablet for tablet 57c1fca980d1415396eb94c3aadd77dd (DEFAULT_TABLE table=test-table [id=6fc3d7714c5e48b0a6ee4b829432debd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:01.971753 24249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c1fca980d1415396eb94c3aadd77dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:01.972019 24118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c1fca980d1415396eb94c3aadd77dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:01.971774 24380 tablet_service.cc:1467] Processing CreateTablet for tablet 57c1fca980d1415396eb94c3aadd77dd (DEFAULT_TABLE table=test-table [id=6fc3d7714c5e48b0a6ee4b829432debd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:01.972685 24380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c1fca980d1415396eb94c3aadd77dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:01.978040 24463 tablet_bootstrap.cc:492] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: Bootstrap starting.
I20260925 00:47:01.979509 24464 tablet_bootstrap.cc:492] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11: Bootstrap starting.
I20260925 00:47:01.979740 24465 tablet_bootstrap.cc:492] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: Bootstrap starting.
I20260925 00:47:01.980391 24463 tablet_bootstrap.cc:654] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:01.981101 24463 log.cc:824] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:01.981645 24465 tablet_bootstrap.cc:654] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:01.981645 24464 tablet_bootstrap.cc:654] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:01.982465 24465 log.cc:824] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:01.982465 24464 log.cc:824] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:01.983124 24463 tablet_bootstrap.cc:492] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: No bootstrap required, opened a new log
I20260925 00:47:01.983325 24463 ts_tablet_manager.cc:1397] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260925 00:47:01.984298 24464 tablet_bootstrap.cc:492] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11: No bootstrap required, opened a new log
I20260925 00:47:01.984298 24465 tablet_bootstrap.cc:492] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: No bootstrap required, opened a new log
I20260925 00:47:01.984467 24464 ts_tablet_manager.cc:1397] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260925 00:47:01.984467 24465 ts_tablet_manager.cc:1397] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 00:47:01.987108 24463 raft_consensus.cc:348] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 } } peers { permanent_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44147 } } peers { permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 } }
I20260925 00:47:01.987350 24463 raft_consensus.cc:374] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:01.987403 24463 raft_consensus.cc:729] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4333820f3f2433abc6cfda5239e18fd, State: Initialized, Role: FOLLOWER
I20260925 00:47:01.987867 24463 consensus_queue.cc:260] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 } } peers { permanent_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44147 } } peers { permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 } }
I20260925 00:47:01.988555 24315 heartbeater.cc:503] Master 127.6.72.126:41691 was elected leader, sending a full tablet report...
I20260925 00:47:01.988749 24464 raft_consensus.cc:348] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 } } peers { permanent_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44147 } } peers { permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 } }
I20260925 00:47:01.988824 24465 raft_consensus.cc:348] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 } } peers { permanent_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44147 } } peers { permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 } }
I20260925 00:47:01.988997 24464 raft_consensus.cc:374] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:01.989008 24465 raft_consensus.cc:374] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:01.989029 24463 ts_tablet_manager.cc:1428] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260925 00:47:01.989048 24464 raft_consensus.cc:729] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bccb3ea5fbd412d9c28b856d20d3f11, State: Initialized, Role: FOLLOWER
I20260925 00:47:01.989073 24465 raft_consensus.cc:729] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301ec7153f5c4fa19d225e93a7eacdd4, State: Initialized, Role: FOLLOWER
I20260925 00:47:01.989465 24465 consensus_queue.cc:260] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 } } peers { permanent_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44147 } } peers { permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 } }
I20260925 00:47:01.989472 24464 consensus_queue.cc:260] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 } } peers { permanent_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44147 } } peers { permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 } }
I20260925 00:47:01.990347 24464 ts_tablet_manager.cc:1428] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 00:47:01.990347 24465 ts_tablet_manager.cc:1428] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 00:47:01.990664 24446 heartbeater.cc:503] Master 127.6.72.126:41691 was elected leader, sending a full tablet report...
I20260925 00:47:01.990693 24184 heartbeater.cc:503] Master 127.6.72.126:41691 was elected leader, sending a full tablet report...
W20260925 00:47:02.015340 24185 tablet.cc:2369] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:47:02.174822 24447 tablet.cc:2369] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:47:02.181372 24470 raft_consensus.cc:482] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:47:02.181593 24470 raft_consensus.cc:504] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 } } peers { permanent_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44147 } } peers { permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 } }
I20260925 00:47:02.182839 24470 leader_election.cc:288] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4333820f3f2433abc6cfda5239e18fd (127.6.72.66:39507), 1bccb3ea5fbd412d9c28b856d20d3f11 (127.6.72.65:39953)
I20260925 00:47:02.187510 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:02.183225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:36237 (local address 127.6.72.65:39953)
0925 00:47:02.183378 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:02.183384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:02.183402 (+ 18us) server_negotiation.cc:407] Connection header received
0925 00:47:02.183486 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:02.183490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:02.183544 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:02.183633 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:02.184022 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:02.184662 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:02.186265 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:02.186882 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:02.186989 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:02.187073 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:02.187269 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 00:47:02.187384 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 00:47:02.187510 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:02.183279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:60645 (local address 127.6.72.66:39507)
0925 00:47:02.183405 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:02.183412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:02.183719 (+ 307us) server_negotiation.cc:407] Connection header received
0925 00:47:02.183857 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:02.183860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:02.183921 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:02.184004 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:02.184312 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:02.184858 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:02.186361 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:02.186991 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:02.187083 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:02.187147 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:02.187334 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 00:47:02.187407 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:47:02.187510 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:02.183079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:39953 (local address 127.6.72.67:36237)
0925 00:47:02.183241 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:02.183258 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:02.183363 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:02.183647 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:02.183649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:02.183663 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:02.183855 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:02.183863 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:02.184775 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:02.184778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:02.186138 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:02.186149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:02.186993 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:02.186995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:02.187122 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:02.187150 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 00:47:02.187241 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 00:47:02.187328 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":90,"spinlock_wait_cycles":10624}
I20260925 00:47:02.187518 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:02.183079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:39507 (local address 127.6.72.67:60645)
0925 00:47:02.183597 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:02.183613 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:02.183740 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:02.184025 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:02.184028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:02.184046 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:02.184162 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:02.184168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:02.185000 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:02.185003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:02.186228 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:02.186238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:02.187098 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:02.187101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:02.187215 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:02.187240 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 00:47:02.187305 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 00:47:02.187352 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":156,"thread_start_us":107,"threads_started":1}
I20260925 00:47:02.188367 24138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c1fca980d1415396eb94c3aadd77dd" candidate_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" is_pre_election: true
I20260925 00:47:02.188369 24269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c1fca980d1415396eb94c3aadd77dd" candidate_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4333820f3f2433abc6cfda5239e18fd" is_pre_election: true
I20260925 00:47:02.188709 24138 raft_consensus.cc:2454] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301ec7153f5c4fa19d225e93a7eacdd4 in term 0.
I20260925 00:47:02.188709 24269 raft_consensus.cc:2454] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301ec7153f5c4fa19d225e93a7eacdd4 in term 0.
I20260925 00:47:02.189253 24335 leader_election.cc:302] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301ec7153f5c4fa19d225e93a7eacdd4, b4333820f3f2433abc6cfda5239e18fd; no voters:
I20260925 00:47:02.189510 24470 raft_consensus.cc:2781] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:47:02.189596 24470 raft_consensus.cc:482] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:47:02.189641 24470 raft_consensus.cc:3037] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:02.190685 24470 raft_consensus.cc:504] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 } } peers { permanent_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44147 } } peers { permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 } }
I20260925 00:47:02.191040 24470 leader_election.cc:288] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [CANDIDATE]: Term 1 election: Requested vote from peers b4333820f3f2433abc6cfda5239e18fd (127.6.72.66:39507), 1bccb3ea5fbd412d9c28b856d20d3f11 (127.6.72.65:39953)
I20260925 00:47:02.191478 24269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c1fca980d1415396eb94c3aadd77dd" candidate_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4333820f3f2433abc6cfda5239e18fd"
I20260925 00:47:02.191591 24138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c1fca980d1415396eb94c3aadd77dd" candidate_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11"
I20260925 00:47:02.191601 24269 raft_consensus.cc:3037] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:02.191740 24138 raft_consensus.cc:3037] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:02.192734 24269 raft_consensus.cc:2454] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301ec7153f5c4fa19d225e93a7eacdd4 in term 1.
I20260925 00:47:02.192929 24138 raft_consensus.cc:2454] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301ec7153f5c4fa19d225e93a7eacdd4 in term 1.
I20260925 00:47:02.193100 24335 leader_election.cc:302] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301ec7153f5c4fa19d225e93a7eacdd4, b4333820f3f2433abc6cfda5239e18fd; no voters:
I20260925 00:47:02.193526 24470 raft_consensus.cc:2781] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:02.193805 24470 raft_consensus.cc:686] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 1 LEADER]: Becoming Leader. State: Replica: 301ec7153f5c4fa19d225e93a7eacdd4, State: Running, Role: LEADER
I20260925 00:47:02.194494 24470 consensus_queue.cc:237] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 } } peers { permanent_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44147 } } peers { permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 } }
I20260925 00:47:02.197805 23996 catalog_manager.cc:5495] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 301ec7153f5c4fa19d225e93a7eacdd4 (127.6.72.67). New cstate: current_term: 1 leader_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301ec7153f5c4fa19d225e93a7eacdd4" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 44147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 } health_report { overall_health: UNKNOWN } } }
W20260925 00:47:02.213577 24316 tablet.cc:2369] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:47:02.259243 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:02.255842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60950 (local address 127.6.72.67:44147)
0925 00:47:02.255998 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:02.256002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:02.256014 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:47:02.256091 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:02.256093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:02.256145 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:02.256294 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:02.256586 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:02.257108 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:02.257924 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:02.258129 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:02.258206 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:02.258480 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:02.258929 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:02.259023 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:02.259073 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 00:47:02.259122 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 00:47:02.265762 24269 raft_consensus.cc:1261] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd [term 1 FOLLOWER]: Refusing update from remote peer 301ec7153f5c4fa19d225e93a7eacdd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:02.265790 24138 raft_consensus.cc:1261] T 57c1fca980d1415396eb94c3aadd77dd P 1bccb3ea5fbd412d9c28b856d20d3f11 [term 1 FOLLOWER]: Refusing update from remote peer 301ec7153f5c4fa19d225e93a7eacdd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:02.266649 24470 consensus_queue.cc:1035] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bccb3ea5fbd412d9c28b856d20d3f11" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 39953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:02.266871 24473 consensus_queue.cc:1035] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4333820f3f2433abc6cfda5239e18fd" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:02.276094 24476 mvcc.cc:205] Tried to move back new op lower bound from 7333057422392471552 to 7333057422111330304. Current Snapshot: MvccSnapshot[applied={T|T < 7333057422392471552}]
I20260925 00:47:02.278247 24478 mvcc.cc:205] Tried to move back new op lower bound from 7333057422392471552 to 7333057422111330304. Current Snapshot: MvccSnapshot[applied={T|T < 7333057422392471552}]
I20260925 00:47:02.278743 24477 mvcc.cc:205] Tried to move back new op lower bound from 7333057422392471552 to 7333057422111330304. Current Snapshot: MvccSnapshot[applied={T|T < 7333057422392471552}]
I20260925 00:47:02.286928 23996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:02.287141 23996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:02.290062 23996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 534eb2657b144da79f9d36a355ff58d8: Sending DeleteTablet for 3 replicas of tablet 57c1fca980d1415396eb94c3aadd77dd
I20260925 00:47:02.291044 24249 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c1fca980d1415396eb94c3aadd77dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:02 UTC) from {username='slave'} at 127.0.0.1:43918
I20260925 00:47:02.291050 24380 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c1fca980d1415396eb94c3aadd77dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:02 UTC) from {username='slave'} at 127.0.0.1:60934
I20260925 00:47:02.291612 24485 tablet_replica.cc:330] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: stopping tablet replica
I20260925 00:47:02.291867 24118 tablet_service.cc:1514] Processing DeleteTablet for tablet 57c1fca980d1415396eb94c3aadd77dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:02 UTC) from {username='slave'} at 127.0.0.1:44706
I20260925 00:47:02.292124 24485 raft_consensus.cc:2229] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:02.292461 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 24056
I20260925 00:47:02.292662 24485 raft_consensus.cc:2258] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:02.293180 24486 tablet_replica.cc:330] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: stopping tablet replica
I20260925 00:47:02.293602 24486 raft_consensus.cc:2229] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:47:02.294130 24486 raft_consensus.cc:2258] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:02.295045 24485 ts_tablet_manager.cc:1905] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:02.295765 24486 ts_tablet_manager.cc:1905] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:02.298092 24485 ts_tablet_manager.cc:1918] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:02.298214 24485 log.cc:1196] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/57c1fca980d1415396eb94c3aadd77dd
I20260925 00:47:02.298532 24485 ts_tablet_manager.cc:1939] T 57c1fca980d1415396eb94c3aadd77dd P b4333820f3f2433abc6cfda5239e18fd: Deleting consensus metadata
I20260925 00:47:02.299458 24486 ts_tablet_manager.cc:1918] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:02.299654 24486 log.cc:1196] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/57c1fca980d1415396eb94c3aadd77dd
I20260925 00:47:02.300045 24486 ts_tablet_manager.cc:1939] T 57c1fca980d1415396eb94c3aadd77dd P 301ec7153f5c4fa19d225e93a7eacdd4: Deleting consensus metadata
I20260925 00:47:02.300051 23982 catalog_manager.cc:4841] TS b4333820f3f2433abc6cfda5239e18fd (127.6.72.66:39507): tablet 57c1fca980d1415396eb94c3aadd77dd (table test-table [id=6fc3d7714c5e48b0a6ee4b829432debd]) successfully deleted
I20260925 00:47:02.301441 23980 catalog_manager.cc:4841] TS 301ec7153f5c4fa19d225e93a7eacdd4 (127.6.72.67:44147): tablet 57c1fca980d1415396eb94c3aadd77dd (table test-table [id=6fc3d7714c5e48b0a6ee4b829432debd]) successfully deleted
W20260925 00:47:02.302028 23980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:39953 (error 108)
I20260925 00:47:02.302168 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 24187
I20260925 00:47:02.304404 24457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:02.304076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:39953 (local address 127.0.0.1:44710)
0925 00:47:02.304215 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:02.304320 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:39953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260925 00:47:02.304801 23980 catalog_manager.cc:4584] TS 1bccb3ea5fbd412d9c28b856d20d3f11 (127.6.72.65:39953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 57c1fca980d1415396eb94c3aadd77dd: Network error: Client connection negotiation failed: client connection to 127.6.72.65:39953: connect: Connection refused (error 111)
I20260925 00:47:02.309405 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 24318
I20260925 00:47:02.314402 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 23965
2026-09-25T00:47:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1270 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:02 dist-test-slave-w8nh krb5kdc[24491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:02 dist-test-slave-w8nh krb5kdc[24491](info): set up 2 sockets
Sep 25 00:47:02 dist-test-slave-w8nh krb5kdc[24491](info): commencing operation
krb5kdc: starting...
W20260925 00:47:03.360113 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:03 dist-test-slave-w8nh krb5kdc[24491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:47:03.482798 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:43603
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:43603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:03.592178 24504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:03.592542 24504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:03.592618 24504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:03.596716 24504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:03.596827 24504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:03.596866 24504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:03.596922 24504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:03.596959 24504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:03.602722 24504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:43603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:43603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:03.603914 24504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:03.604997 24504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:03.613659 24509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:03.613662 24512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:03.613683 24510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:03.614033 24504 server_base.cc:1034] running on GCE node
I20260925 00:47:03.614563 24504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:03.615686 24504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:03.616852 24504 hybrid_clock.cc:648] HybridClock initialized: now 1790297223616833 us; error 28 us; skew 500 ppm
Sep 25 00:47:03 dist-test-slave-w8nh krb5kdc[24491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297223, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:03.619606 24504 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:47:03.620645 24504 webserver.cc:460] Webserver started at http://127.6.72.126:33323/ using document root <none> and password file <none>
I20260925 00:47:03.621227 24504 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:03.621320 24504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:03.621555 24504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:03.623629 24504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "b487ce6015f74939b7a3888049a238e3"
format_stamp: "Formatted at 2026-09-25 00:47:03 on dist-test-slave-w8nh"
server_key: "f6b78f2c6cf451973632567a5080246f"
server_key_iv: "f3828bb1f42c9ab54817132f6f449e7e"
server_key_version: "encryptionkey@0"
I20260925 00:47:03.624217 24504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "b487ce6015f74939b7a3888049a238e3"
format_stamp: "Formatted at 2026-09-25 00:47:03 on dist-test-slave-w8nh"
server_key: "f6b78f2c6cf451973632567a5080246f"
server_key_iv: "f3828bb1f42c9ab54817132f6f449e7e"
server_key_version: "encryptionkey@0"
I20260925 00:47:03.628037 24504 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:03.630733 24519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:03.632094 24504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:47:03.632316 24504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "b487ce6015f74939b7a3888049a238e3"
format_stamp: "Formatted at 2026-09-25 00:47:03 on dist-test-slave-w8nh"
server_key: "f6b78f2c6cf451973632567a5080246f"
server_key_iv: "f3828bb1f42c9ab54817132f6f449e7e"
server_key_version: "encryptionkey@0"
I20260925 00:47:03.632452 24504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:03.657760 24504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:03.660858 24504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:03.661062 24504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:03.668009 24504 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:43603
I20260925 00:47:03.668033 24571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:43603 every 8 connection(s)
I20260925 00:47:03.669029 24504 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:03.669229 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 24504
I20260925 00:47:03.669390 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:03.669772 6433 external_mini_cluster.cc:1428] Setting key dc9da50646de7bbd1c187c507aaa0e45
I20260925 00:47:03.672364 24572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:03.677960 24572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3: Bootstrap starting.
Sep 25 00:47:03 dist-test-slave-w8nh krb5kdc[24491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:03.680615 24572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:03.681363 24572 log.cc:824] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:03.683408 24572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3: No bootstrap required, opened a new log
I20260925 00:47:03.683977 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:03.671202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56084 (local address 127.6.72.126:43603)
0925 00:47:03.671686 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:03.671705 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:03.671742 (+ 37us) server_negotiation.cc:407] Connection header received
0925 00:47:03.672482 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:03.672506 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:03.672851 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:03.673199 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:03.674021 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:03.675016 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:03.675900 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:03.676453 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:03.676624 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:03.679117 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:03.679122 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:03.679136 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:03.679164 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:03.681050 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:03.681512 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:03.681520 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:03.681531 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:03.681643 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:03.681935 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:03.681938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:03.681940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:03.682344 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:03.682548 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:03.682864 (+ 316us) server_negotiation.cc:299] Negotiation successful
0925 00:47:03.683158 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":160,"threads_started":1}
I20260925 00:47:03.687188 24572 raft_consensus.cc:348] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487ce6015f74939b7a3888049a238e3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43603 } }
I20260925 00:47:03.687500 24572 raft_consensus.cc:374] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:03.687582 24572 raft_consensus.cc:729] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b487ce6015f74939b7a3888049a238e3, State: Initialized, Role: FOLLOWER
I20260925 00:47:03.688017 24572 consensus_queue.cc:260] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487ce6015f74939b7a3888049a238e3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43603 } }
I20260925 00:47:03.688200 24572 raft_consensus.cc:388] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:03.688326 24572 raft_consensus.cc:482] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:03.688441 24572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:03.689549 24572 raft_consensus.cc:504] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487ce6015f74939b7a3888049a238e3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43603 } }
I20260925 00:47:03.689854 24572 leader_election.cc:302] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b487ce6015f74939b7a3888049a238e3; no voters:
I20260925 00:47:03.690117 24572 leader_election.cc:288] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:03.690263 24577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:03.690505 24577 raft_consensus.cc:686] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [term 1 LEADER]: Becoming Leader. State: Replica: b487ce6015f74939b7a3888049a238e3, State: Running, Role: LEADER
I20260925 00:47:03.690866 24577 consensus_queue.cc:237] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487ce6015f74939b7a3888049a238e3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43603 } }
I20260925 00:47:03.691382 24572 sys_catalog.cc:564] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:03.692525 24578 sys_catalog.cc:455] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b487ce6015f74939b7a3888049a238e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487ce6015f74939b7a3888049a238e3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43603 } } }
I20260925 00:47:03.692672 24578 sys_catalog.cc:458] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:03.692853 24579 sys_catalog.cc:455] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b487ce6015f74939b7a3888049a238e3. Latest consensus state: current_term: 1 leader_uuid: "b487ce6015f74939b7a3888049a238e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b487ce6015f74939b7a3888049a238e3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43603 } } }
I20260925 00:47:03.693106 24579 sys_catalog.cc:458] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3 [sys.catalog]: This master's current role is: LEADER
W20260925 00:47:03.696313 24592 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:03.696404 24592 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:03.696481 24587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:03.697191 24587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:03.703007 24587 catalog_manager.cc:1347] Generated new cluster ID: dd8fd32302b54e74b9ca1bd04b736f15
I20260925 00:47:03.703088 24587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:03.719262 24587 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:03.720216 24587 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:03.726114 24587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3: Generated new TSK 0
I20260925 00:47:03.726804 24587 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:47:03.755067 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:43603
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:03.866772 24598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:03.867053 24598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:03.867096 24598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:03.871181 24598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:03.871243 24598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:03.871302 24598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:47:03.876557 24598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:43603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:03.877565 24598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:03.878531 24598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:03.889039 24603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:03.889083 24606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:03.889039 24604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:03.889376 24598 server_base.cc:1034] running on GCE node
I20260925 00:47:03.889766 24598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:03.890409 24598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:03.891615 24598 hybrid_clock.cc:648] HybridClock initialized: now 1790297223891595 us; error 42 us; skew 500 ppm
Sep 25 00:47:03 dist-test-slave-w8nh krb5kdc[24491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297223, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:03.894521 24598 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:47:03.895411 24598 webserver.cc:460] Webserver started at http://127.6.72.65:37715/ using document root <none> and password file <none>
I20260925 00:47:03.895994 24598 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:03.896086 24598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:03.896392 24598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:03.898496 24598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "a9da5c2b05fe4d25a118251822163f14"
format_stamp: "Formatted at 2026-09-25 00:47:03 on dist-test-slave-w8nh"
server_key: "63d796f91c309ce4faa95b5c11ec3026"
server_key_iv: "f2a1d477200143ac93b83d548d8d96f4"
server_key_version: "encryptionkey@0"
I20260925 00:47:03.899153 24598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "a9da5c2b05fe4d25a118251822163f14"
format_stamp: "Formatted at 2026-09-25 00:47:03 on dist-test-slave-w8nh"
server_key: "63d796f91c309ce4faa95b5c11ec3026"
server_key_iv: "f2a1d477200143ac93b83d548d8d96f4"
server_key_version: "encryptionkey@0"
I20260925 00:47:03.903475 24598 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260925 00:47:03.906500 24613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:03.907823 24598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 00:47:03.907984 24598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "a9da5c2b05fe4d25a118251822163f14"
format_stamp: "Formatted at 2026-09-25 00:47:03 on dist-test-slave-w8nh"
server_key: "63d796f91c309ce4faa95b5c11ec3026"
server_key_iv: "f2a1d477200143ac93b83d548d8d96f4"
server_key_version: "encryptionkey@0"
I20260925 00:47:03.908121 24598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:03.941752 24598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:03.945384 24598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:03.945654 24598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:03.946326 24598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:03.947274 24598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:03.947350 24598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:03.947430 24598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:03.947486 24598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:03.958460 24598 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:41519
I20260925 00:47:03.958503 24726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:41519 every 8 connection(s)
I20260925 00:47:03.959494 24598 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:47:03.961926 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 24598
I20260925 00:47:03.962067 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:47:03.962433 6433 external_mini_cluster.cc:1428] Setting key 49fdbcd3361ab6ced08371763bc61a0c
Sep 25 00:47:03 dist-test-slave-w8nh krb5kdc[24491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297223, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:47:03.976281 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:03.961668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:34911 (local address 127.6.72.126:43603)
0925 00:47:03.961850 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:03.961854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:03.963053 (+ 1199us) server_negotiation.cc:407] Connection header received
0925 00:47:03.964073 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:03.964077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:03.964134 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:03.964295 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:03.966044 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:03.966947 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:03.968174 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:03.968688 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:03.968826 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:03.972063 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:03.972069 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:03.972073 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:03.972109 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:03.974024 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:03.974597 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:03.974600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:03.974602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:03.974690 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:03.975052 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:03.975056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:03.975058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:03.975265 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:03.975378 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:03.975920 (+ 542us) server_negotiation.cc:299] Negotiation successful
0925 00:47:03.976067 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 00:47:03.977144 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:03.962210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:43603 (local address 127.6.72.65:34911)
0925 00:47:03.962826 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:03.962905 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:03.963839 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:03.964490 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:03.964500 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:03.965080 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:03.965837 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:03.965860 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:03.967101 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:03.967103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:03.968029 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:03.968038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:03.968879 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:03.968882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:03.969048 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:03.969812 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:03.969839 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:03.971857 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:03.974169 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:03.974176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:03.974198 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:03.974484 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:03.974790 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:03.974794 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:03.974796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:03.974953 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:03.975390 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:03.975399 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:03.975671 (+ 272us) client_negotiation.cc:769] Sending connection context
0925 00:47:03.975879 (+ 208us) client_negotiation.cc:240] Negotiation successful
0925 00:47:03.976242 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":159,"threads_started":1}
I20260925 00:47:03.978593 24727 heartbeater.cc:347] Connected to a master server at 127.6.72.126:43603
I20260925 00:47:03.978881 24727 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:03.979435 24727 heartbeater.cc:511] Master 127.6.72.126:43603 requested a full tablet report, sending...
I20260925 00:47:03.981166 24534 ts_manager.cc:194] Registered new tserver with Master: a9da5c2b05fe4d25a118251822163f14 (127.6.72.65:41519)
I20260925 00:47:03.982332 24534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:34911
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:47:03.993759 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:43603
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:04.111007 24732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:04.111413 24732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:04.111517 24732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:04.115959 24732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:04.116086 24732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:04.116199 24732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:47:04.121951 24732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:43603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:04.123154 24732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:04.124225 24732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:04.135140 24740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:04.135138 24737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:04.135291 24732 server_base.cc:1034] running on GCE node
W20260925 00:47:04.135486 24738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:04.135910 24732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:04.136665 24732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:04.137835 24732 hybrid_clock.cc:648] HybridClock initialized: now 1790297224137819 us; error 28 us; skew 500 ppm
Sep 25 00:47:04 dist-test-slave-w8nh krb5kdc[24491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297224, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:04.140937 24732 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:47:04.141922 24732 webserver.cc:460] Webserver started at http://127.6.72.66:37393/ using document root <none> and password file <none>
I20260925 00:47:04.142518 24732 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:04.142614 24732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:04.142854 24732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:04.145059 24732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "b16d7f7338f5477ba29ab1b58f3bc36e"
format_stamp: "Formatted at 2026-09-25 00:47:04 on dist-test-slave-w8nh"
server_key: "724785d8fc706dac7253a629c63c0d35"
server_key_iv: "0fa4b0b6c7ec661532877a0448f44b1b"
server_key_version: "encryptionkey@0"
I20260925 00:47:04.145656 24732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "b16d7f7338f5477ba29ab1b58f3bc36e"
format_stamp: "Formatted at 2026-09-25 00:47:04 on dist-test-slave-w8nh"
server_key: "724785d8fc706dac7253a629c63c0d35"
server_key_iv: "0fa4b0b6c7ec661532877a0448f44b1b"
server_key_version: "encryptionkey@0"
I20260925 00:47:04.149580 24732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:47:04.152180 24747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:04.153631 24732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:47:04.153782 24732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "b16d7f7338f5477ba29ab1b58f3bc36e"
format_stamp: "Formatted at 2026-09-25 00:47:04 on dist-test-slave-w8nh"
server_key: "724785d8fc706dac7253a629c63c0d35"
server_key_iv: "0fa4b0b6c7ec661532877a0448f44b1b"
server_key_version: "encryptionkey@0"
I20260925 00:47:04.153925 24732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:04.168460 24732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:04.171521 24732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:04.171756 24732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:04.172416 24732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:04.173313 24732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:04.173393 24732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:04.173466 24732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:04.173518 24732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:04.182956 24732 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:33245
I20260925 00:47:04.182978 24860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:33245 every 8 connection(s)
I20260925 00:47:04.184052 24732 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:47:04.190830 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 24732
I20260925 00:47:04.190965 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:47:04.191278 6433 external_mini_cluster.cc:1428] Setting key 586daff2d65a478658798c03ec16271f
Sep 25 00:47:04 dist-test-slave-w8nh krb5kdc[24491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297224, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:04.198695 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.185993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:49221 (local address 127.6.72.126:43603)
0925 00:47:04.186142 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:04.186146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:04.187239 (+ 1093us) server_negotiation.cc:407] Connection header received
0925 00:47:04.188168 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:04.188170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:04.188228 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:04.188353 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:04.189568 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.190093 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.191609 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.191889 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.191972 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.194921 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:04.194924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:04.194927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:04.194955 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:04.196496 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:04.197091 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:04.197095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:04.197096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:04.197182 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:04.197535 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:04.197539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:04.197541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:04.197750 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:04.197859 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:04.198393 (+ 534us) server_negotiation.cc:299] Negotiation successful
0925 00:47:04.198527 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:47:04.199600 24863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.186363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:43603 (local address 127.6.72.66:49221)
0925 00:47:04.187074 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:04.187118 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:04.187955 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:04.188543 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:04.188550 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:04.188947 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:04.189431 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.189444 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.190206 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.190208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.191453 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.191465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.192022 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.192026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.192215 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.192974 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:04.192991 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:04.194736 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:04.196621 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:04.196627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:04.196646 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:04.196975 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:04.197278 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:04.197282 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:04.197284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:04.197434 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:04.197859 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:04.197868 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:04.198138 (+ 270us) client_negotiation.cc:769] Sending connection context
0925 00:47:04.198348 (+ 210us) client_negotiation.cc:240] Negotiation successful
0925 00:47:04.198706 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":110,"threads_started":1}
I20260925 00:47:04.201077 24861 heartbeater.cc:347] Connected to a master server at 127.6.72.126:43603
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
I20260925 00:47:04.201344 24861 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:04.201859 24861 heartbeater.cc:511] Master 127.6.72.126:43603 requested a full tablet report, sending...
I20260925 00:47:04.203049 24534 ts_manager.cc:194] Registered new tserver with Master: b16d7f7338f5477ba29ab1b58f3bc36e (127.6.72.66:33245)
I20260925 00:47:04.203675 24534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:49221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:47:04.220347 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:43603
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:04.339429 24866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:04.339747 24866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:04.339831 24866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:04.344194 24866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:04.344359 24866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:04.344487 24866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:47:04.350250 24866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:43603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:04.351434 24866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:04.352617 24866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:04.362865 24872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:04.362865 24871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:04.362881 24874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:04.363305 24866 server_base.cc:1034] running on GCE node
I20260925 00:47:04.363785 24866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:04.364537 24866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:04.365736 24866 hybrid_clock.cc:648] HybridClock initialized: now 1790297224365698 us; error 49 us; skew 500 ppm
Sep 25 00:47:04 dist-test-slave-w8nh krb5kdc[24491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297224, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:04.368768 24866 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:47:04.369678 24866 webserver.cc:460] Webserver started at http://127.6.72.67:37787/ using document root <none> and password file <none>
I20260925 00:47:04.370239 24866 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:04.370303 24866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:04.370479 24866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:04.372673 24866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "30a0328992664a46b2a0f294cc326cab"
format_stamp: "Formatted at 2026-09-25 00:47:04 on dist-test-slave-w8nh"
server_key: "cf2e375549785c78d0ae70ecb1e7b62c"
server_key_iv: "8df9ebf866d586d09d653a9b8307d26a"
server_key_version: "encryptionkey@0"
I20260925 00:47:04.373240 24866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "30a0328992664a46b2a0f294cc326cab"
format_stamp: "Formatted at 2026-09-25 00:47:04 on dist-test-slave-w8nh"
server_key: "cf2e375549785c78d0ae70ecb1e7b62c"
server_key_iv: "8df9ebf866d586d09d653a9b8307d26a"
server_key_version: "encryptionkey@0"
I20260925 00:47:04.377228 24866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:47:04.379833 24881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:04.381328 24866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:04.381434 24866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "30a0328992664a46b2a0f294cc326cab"
format_stamp: "Formatted at 2026-09-25 00:47:04 on dist-test-slave-w8nh"
server_key: "cf2e375549785c78d0ae70ecb1e7b62c"
server_key_iv: "8df9ebf866d586d09d653a9b8307d26a"
server_key_version: "encryptionkey@0"
I20260925 00:47:04.381623 24866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:04.416828 24866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:04.420166 24866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:04.420517 24866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:04.421257 24866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:04.422199 24866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:04.422287 24866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:04.422384 24866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:04.422426 24866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:04.432296 24994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:39395 every 8 connection(s)
I20260925 00:47:04.432302 24866 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:39395
I20260925 00:47:04.433306 24866 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:47:04.438210 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 24866
I20260925 00:47:04.438382 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:47:04.438766 6433 external_mini_cluster.cc:1428] Setting key e5041d7f63527652fa845ac69bcd9c06
Sep 25 00:47:04 dist-test-slave-w8nh krb5kdc[24491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297224, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:04.448741 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.435403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:46551 (local address 127.6.72.126:43603)
0925 00:47:04.435554 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:04.435558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:04.436787 (+ 1229us) server_negotiation.cc:407] Connection header received
0925 00:47:04.437723 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:04.437726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:04.437774 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:04.437866 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:04.439622 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.440452 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.441888 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.442133 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.442225 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.445099 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:04.445103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:04.445110 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:04.445136 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:04.446578 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:04.447114 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:04.447118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:04.447120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:04.447205 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:04.447549 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:04.447554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:04.447556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:04.447748 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:04.447852 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:04.448431 (+ 579us) server_negotiation.cc:299] Negotiation successful
0925 00:47:04.448561 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 00:47:04.449620 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.435822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:43603 (local address 127.6.72.67:46551)
0925 00:47:04.436583 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:04.436643 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:04.437522 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:04.438058 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:04.438070 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:04.438645 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:04.439352 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.439375 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.440629 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.440631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.441742 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.441753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.442297 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.442300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.442477 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.443153 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:04.443171 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:04.444913 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:04.446713 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:04.446719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:04.446736 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:04.447004 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:04.447298 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:04.447301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:04.447303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:04.447453 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:04.447850 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:04.447857 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:04.448119 (+ 262us) client_negotiation.cc:769] Sending connection context
0925 00:47:04.448393 (+ 274us) client_negotiation.cc:240] Negotiation successful
0925 00:47:04.448741 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":114,"threads_started":1}
I20260925 00:47:04.450781 24995 heartbeater.cc:347] Connected to a master server at 127.6.72.126:43603
I20260925 00:47:04.451046 24995 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:04.451599 24995 heartbeater.cc:511] Master 127.6.72.126:43603 requested a full tablet report, sending...
I20260925 00:47:04.452621 24534 ts_manager.cc:194] Registered new tserver with Master: 30a0328992664a46b2a0f294cc326cab (127.6.72.67:39395)
I20260925 00:47:04.453091 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:47:04.453281 24534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:46551
I20260925 00:47:04.463382 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.455451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.6.72.126:43603)
0925 00:47:04.455576 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:04.455579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:04.455733 (+ 154us) server_negotiation.cc:407] Connection header received
0925 00:47:04.455835 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:04.455837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:04.455870 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:04.455932 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:04.456753 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.457332 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.458422 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.458630 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.458719 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.460473 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:04.460476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:04.460479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:04.460500 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:04.461912 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:04.462354 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:04.462357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:04.462358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:04.462427 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:04.462698 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:04.462700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:04.462701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:04.462891 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:04.463003 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:04.463185 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 00:47:04.463262 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 00:47:04.466214 24534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:47:04.469213 24534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:47:04.483364 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.477337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:33245 (local address 127.0.0.1:47588)
0925 00:47:04.477863 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:04.477910 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:04.478040 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:04.478732 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:04.478738 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:04.478765 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:04.478926 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.478934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.480463 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.480467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.481645 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.481657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.482777 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.482781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.482899 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.482955 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 00:47:04.483059 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 00:47:04.483189 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"spinlock_wait_cycles":768,"thread_start_us":107,"threads_started":1}
I20260925 00:47:04.483541 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.478138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:39395 (local address 127.0.0.1:39406)
0925 00:47:04.478695 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:04.478709 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:04.478809 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:04.479410 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:04.479413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:04.479431 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:04.479522 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.479527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.480811 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.480816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.482072 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.482082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.483022 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.483026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.483113 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.483153 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 00:47:04.483237 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 00:47:04.483307 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":33,"thread_start_us":105,"threads_started":1}
I20260925 00:47:04.483735 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.477438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47588 (local address 127.6.72.66:33245)
0925 00:47:04.478118 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:04.478131 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:04.478154 (+ 23us) server_negotiation.cc:407] Connection header received
0925 00:47:04.478234 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:04.478238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:04.478438 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:04.478597 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:04.479066 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.480309 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.481793 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.482629 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.482743 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.482890 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:04.483459 (+ 569us) server_negotiation.cc:299] Negotiation successful
0925 00:47:04.483561 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":141,"threads_started":1}
I20260925 00:47:04.483747 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.477421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:41519 (local address 127.0.0.1:49400)
0925 00:47:04.478421 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:04.478435 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:04.478527 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:04.479608 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:04.479611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:04.479708 (+ 97us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:04.479809 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.479814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.481004 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.481007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.482195 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.482203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.483265 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.483268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.483364 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.483397 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 00:47:04.483475 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 00:47:04.483547 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"mutex_wait_us":11,"thread_start_us":114,"threads_started":1}
I20260925 00:47:04.484123 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.478420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49400 (local address 127.6.72.65:41519)
0925 00:47:04.479186 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:04.479200 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:04.479222 (+ 22us) server_negotiation.cc:407] Connection header received
0925 00:47:04.479292 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:04.479302 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:04.479467 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:04.479632 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:04.479917 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.480895 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.482318 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.483091 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.483244 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.483373 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:04.483889 (+ 516us) server_negotiation.cc:299] Negotiation successful
0925 00:47:04.483982 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":82,"threads_started":1}
I20260925 00:47:04.485080 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.478576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39406 (local address 127.6.72.67:39395)
0925 00:47:04.478935 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:04.478946 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:04.478968 (+ 22us) server_negotiation.cc:407] Connection header received
0925 00:47:04.479044 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:04.479050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:04.479249 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:04.479391 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:04.479651 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.480689 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.482216 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.482897 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.483005 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.483153 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:04.484777 (+ 1624us) server_negotiation.cc:299] Negotiation successful
0925 00:47:04.484927 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":165,"threads_started":1}
I20260925 00:47:04.486967 24661 tablet_service.cc:1467] Processing CreateTablet for tablet 12537c14edc6424f8c36465be4fa8600 (DEFAULT_TABLE table=test-table [id=128ec2c7902544d381120424f043bd4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:04.486969 24795 tablet_service.cc:1467] Processing CreateTablet for tablet 12537c14edc6424f8c36465be4fa8600 (DEFAULT_TABLE table=test-table [id=128ec2c7902544d381120424f043bd4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:04.487742 24929 tablet_service.cc:1467] Processing CreateTablet for tablet 12537c14edc6424f8c36465be4fa8600 (DEFAULT_TABLE table=test-table [id=128ec2c7902544d381120424f043bd4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:04.488128 24795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12537c14edc6424f8c36465be4fa8600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:04.488128 24661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12537c14edc6424f8c36465be4fa8600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:04.488684 24929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12537c14edc6424f8c36465be4fa8600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:04.495270 25012 tablet_bootstrap.cc:492] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: Bootstrap starting.
I20260925 00:47:04.496114 25014 tablet_bootstrap.cc:492] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: Bootstrap starting.
I20260925 00:47:04.497642 25012 tablet_bootstrap.cc:654] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:04.498239 25014 tablet_bootstrap.cc:654] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:04.498760 25012 log.cc:824] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:04.498947 25014 log.cc:824] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:04.499540 25013 tablet_bootstrap.cc:492] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14: Bootstrap starting.
I20260925 00:47:04.501098 25014 tablet_bootstrap.cc:492] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: No bootstrap required, opened a new log
I20260925 00:47:04.501098 25012 tablet_bootstrap.cc:492] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: No bootstrap required, opened a new log
I20260925 00:47:04.501298 25014 ts_tablet_manager.cc:1397] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 00:47:04.501298 25012 ts_tablet_manager.cc:1397] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 00:47:04.501552 25013 tablet_bootstrap.cc:654] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:04.502206 25013 log.cc:824] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:04.503902 25013 tablet_bootstrap.cc:492] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14: No bootstrap required, opened a new log
I20260925 00:47:04.504106 25013 ts_tablet_manager.cc:1397] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 00:47:04.505504 25014 raft_consensus.cc:348] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 } } peers { permanent_uuid: "a9da5c2b05fe4d25a118251822163f14" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41519 } } peers { permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 } }
I20260925 00:47:04.505504 25012 raft_consensus.cc:348] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 } } peers { permanent_uuid: "a9da5c2b05fe4d25a118251822163f14" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41519 } } peers { permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 } }
I20260925 00:47:04.505815 25014 raft_consensus.cc:374] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:04.505815 25012 raft_consensus.cc:374] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:04.505878 25014 raft_consensus.cc:729] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b16d7f7338f5477ba29ab1b58f3bc36e, State: Initialized, Role: FOLLOWER
I20260925 00:47:04.505878 25012 raft_consensus.cc:729] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a0328992664a46b2a0f294cc326cab, State: Initialized, Role: FOLLOWER
I20260925 00:47:04.506376 25014 consensus_queue.cc:260] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 } } peers { permanent_uuid: "a9da5c2b05fe4d25a118251822163f14" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41519 } } peers { permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 } }
I20260925 00:47:04.506376 25012 consensus_queue.cc:260] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 } } peers { permanent_uuid: "a9da5c2b05fe4d25a118251822163f14" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41519 } } peers { permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 } }
I20260925 00:47:04.507472 25012 ts_tablet_manager.cc:1428] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:47:04.507472 25014 ts_tablet_manager.cc:1428] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:47:04.507869 24861 heartbeater.cc:503] Master 127.6.72.126:43603 was elected leader, sending a full tablet report...
I20260925 00:47:04.508430 24995 heartbeater.cc:503] Master 127.6.72.126:43603 was elected leader, sending a full tablet report...
I20260925 00:47:04.508366 25013 raft_consensus.cc:348] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 } } peers { permanent_uuid: "a9da5c2b05fe4d25a118251822163f14" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41519 } } peers { permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 } }
I20260925 00:47:04.508632 25013 raft_consensus.cc:374] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:04.508704 25013 raft_consensus.cc:729] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9da5c2b05fe4d25a118251822163f14, State: Initialized, Role: FOLLOWER
I20260925 00:47:04.509191 25013 consensus_queue.cc:260] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 } } peers { permanent_uuid: "a9da5c2b05fe4d25a118251822163f14" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41519 } } peers { permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 } }
I20260925 00:47:04.510208 25013 ts_tablet_manager.cc:1428] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 00:47:04.510547 24727 heartbeater.cc:503] Master 127.6.72.126:43603 was elected leader, sending a full tablet report...
I20260925 00:47:04.578637 25020 raft_consensus.cc:482] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:47:04.578894 25020 raft_consensus.cc:504] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 } } peers { permanent_uuid: "a9da5c2b05fe4d25a118251822163f14" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41519 } } peers { permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 } }
I20260925 00:47:04.580140 25020 leader_election.cc:288] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b16d7f7338f5477ba29ab1b58f3bc36e (127.6.72.66:33245), 30a0328992664a46b2a0f294cc326cab (127.6.72.67:39395)
I20260925 00:47:04.584477 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.580536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:49679 (local address 127.6.72.66:33245)
0925 00:47:04.580695 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:04.580710 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:04.581123 (+ 413us) server_negotiation.cc:407] Connection header received
0925 00:47:04.581232 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:04.581234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:04.581273 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:04.581325 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:04.581653 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.582171 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.583430 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.583848 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.583969 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.584036 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:04.584265 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 00:47:04.584346 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 00:47:04.584484 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.580537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:34989 (local address 127.6.72.67:39395)
0925 00:47:04.580688 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:04.580692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:04.580962 (+ 270us) server_negotiation.cc:407] Connection header received
0925 00:47:04.581065 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:04.581071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:04.581132 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:04.581229 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:04.581675 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.582170 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.583438 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.583864 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.583984 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.584044 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:04.584283 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 00:47:04.584355 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 00:47:04.584512 25022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.580463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:39395 (local address 127.6.72.65:34989)
0925 00:47:04.580850 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:04.580863 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:04.580970 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:04.581415 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:04.581417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:04.581442 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:04.581543 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.581547 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.582292 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.582298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.583299 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.583308 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.584001 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.584005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.584129 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.584148 (+ 19us) client_negotiation.cc:769] Sending connection context
0925 00:47:04.584276 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 00:47:04.584350 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":74,"thread_start_us":133,"threads_started":1}
I20260925 00:47:04.584774 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.580540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:33245 (local address 127.6.72.65:49679)
0925 00:47:04.581018 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:04.581033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:04.581134 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:04.581374 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:04.581376 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:04.581390 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:04.581524 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.581529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.582299 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.582305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.583296 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:04.583306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.583978 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.583981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:04.584081 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.584129 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 00:47:04.584220 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 00:47:04.584316 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":77,"spinlock_wait_cycles":17152,"thread_start_us":117,"threads_started":1}
I20260925 00:47:04.585255 24949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12537c14edc6424f8c36465be4fa8600" candidate_uuid: "a9da5c2b05fe4d25a118251822163f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a0328992664a46b2a0f294cc326cab" is_pre_election: true
I20260925 00:47:04.585585 24949 raft_consensus.cc:2454] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9da5c2b05fe4d25a118251822163f14 in term 0.
I20260925 00:47:04.585561 24815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12537c14edc6424f8c36465be4fa8600" candidate_uuid: "a9da5c2b05fe4d25a118251822163f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" is_pre_election: true
I20260925 00:47:04.585876 24815 raft_consensus.cc:2454] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9da5c2b05fe4d25a118251822163f14 in term 0.
I20260925 00:47:04.586146 24615 leader_election.cc:302] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a0328992664a46b2a0f294cc326cab, a9da5c2b05fe4d25a118251822163f14; no voters:
I20260925 00:47:04.586442 25020 raft_consensus.cc:2781] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:47:04.586529 25020 raft_consensus.cc:482] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:47:04.586572 25020 raft_consensus.cc:3037] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:04.587653 25020 raft_consensus.cc:504] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 } } peers { permanent_uuid: "a9da5c2b05fe4d25a118251822163f14" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41519 } } peers { permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 } }
I20260925 00:47:04.588018 25020 leader_election.cc:288] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [CANDIDATE]: Term 1 election: Requested vote from peers b16d7f7338f5477ba29ab1b58f3bc36e (127.6.72.66:33245), 30a0328992664a46b2a0f294cc326cab (127.6.72.67:39395)
I20260925 00:47:04.588387 24815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12537c14edc6424f8c36465be4fa8600" candidate_uuid: "a9da5c2b05fe4d25a118251822163f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e"
I20260925 00:47:04.588438 24949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12537c14edc6424f8c36465be4fa8600" candidate_uuid: "a9da5c2b05fe4d25a118251822163f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a0328992664a46b2a0f294cc326cab"
I20260925 00:47:04.588533 24815 raft_consensus.cc:3037] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:04.588572 24949 raft_consensus.cc:3037] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:04.589614 24815 raft_consensus.cc:2454] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9da5c2b05fe4d25a118251822163f14 in term 1.
I20260925 00:47:04.589624 24949 raft_consensus.cc:2454] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9da5c2b05fe4d25a118251822163f14 in term 1.
I20260925 00:47:04.589936 24617 leader_election.cc:302] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9da5c2b05fe4d25a118251822163f14, b16d7f7338f5477ba29ab1b58f3bc36e; no voters:
I20260925 00:47:04.590129 25020 raft_consensus.cc:2781] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:04.590387 25020 raft_consensus.cc:686] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [term 1 LEADER]: Becoming Leader. State: Replica: a9da5c2b05fe4d25a118251822163f14, State: Running, Role: LEADER
I20260925 00:47:04.590767 25020 consensus_queue.cc:237] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 } } peers { permanent_uuid: "a9da5c2b05fe4d25a118251822163f14" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41519 } } peers { permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 } }
I20260925 00:47:04.594174 24536 catalog_manager.cc:5495] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9da5c2b05fe4d25a118251822163f14 (127.6.72.65). New cstate: current_term: 1 leader_uuid: "a9da5c2b05fe4d25a118251822163f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9da5c2b05fe4d25a118251822163f14" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 } health_report { overall_health: UNKNOWN } } }
I20260925 00:47:04.608103 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.604714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.6.72.65:41519)
0925 00:47:04.604846 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:04.604849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:04.604863 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:47:04.604935 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:04.604937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:04.604989 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:04.605092 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:04.605345 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.605887 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.606719 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.606953 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.607042 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.607291 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:04.607749 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:04.607861 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:04.607942 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 00:47:04.607978 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 00:47:04.614254 24815 raft_consensus.cc:1261] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e [term 1 FOLLOWER]: Refusing update from remote peer a9da5c2b05fe4d25a118251822163f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:04.614253 24949 raft_consensus.cc:1261] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab [term 1 FOLLOWER]: Refusing update from remote peer a9da5c2b05fe4d25a118251822163f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:04.614966 25020 consensus_queue.cc:1035] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30a0328992664a46b2a0f294cc326cab" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:04.615164 25023 consensus_queue.cc:1035] T 12537c14edc6424f8c36465be4fa8600 P a9da5c2b05fe4d25a118251822163f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b16d7f7338f5477ba29ab1b58f3bc36e" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 33245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:04.624008 25027 mvcc.cc:205] Tried to move back new op lower bound from 7333057432012505088 to 7333057431926583296. Current Snapshot: MvccSnapshot[applied={T|T < 7333057432012505088}]
I20260925 00:47:04.630997 24534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:04.631161 24534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:04.633646 24534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b487ce6015f74939b7a3888049a238e3: Sending DeleteTablet for 3 replicas of tablet 12537c14edc6424f8c36465be4fa8600
I20260925 00:47:04.634452 24795 tablet_service.cc:1514] Processing DeleteTablet for tablet 12537c14edc6424f8c36465be4fa8600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:04 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:47588
I20260925 00:47:04.634483 24661 tablet_service.cc:1514] Processing DeleteTablet for tablet 12537c14edc6424f8c36465be4fa8600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:04 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:49400
I20260925 00:47:04.634560 24929 tablet_service.cc:1514] Processing DeleteTablet for tablet 12537c14edc6424f8c36465be4fa8600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:04 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:39406
I20260925 00:47:04.635128 25035 tablet_replica.cc:330] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: stopping tablet replica
I20260925 00:47:04.635203 25037 tablet_replica.cc:330] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: stopping tablet replica
I20260925 00:47:04.635550 25035 raft_consensus.cc:2229] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:04.635552 25037 raft_consensus.cc:2229] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:04.635775 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 24598
I20260925 00:47:04.635900 25037 raft_consensus.cc:2258] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:04.637415 25035 raft_consensus.cc:2258] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:04.638887 25037 ts_tablet_manager.cc:1905] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:04.638887 25035 ts_tablet_manager.cc:1905] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:04.641889 25035 ts_tablet_manager.cc:1918] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:04.641889 25037 ts_tablet_manager.cc:1918] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:04.642006 25037 log.cc:1196] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/12537c14edc6424f8c36465be4fa8600
I20260925 00:47:04.642006 25035 log.cc:1196] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/12537c14edc6424f8c36465be4fa8600
I20260925 00:47:04.642374 25037 ts_tablet_manager.cc:1939] T 12537c14edc6424f8c36465be4fa8600 P 30a0328992664a46b2a0f294cc326cab: Deleting consensus metadata
I20260925 00:47:04.642374 25035 ts_tablet_manager.cc:1939] T 12537c14edc6424f8c36465be4fa8600 P b16d7f7338f5477ba29ab1b58f3bc36e: Deleting consensus metadata
W20260925 00:47:04.643554 24521 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:41519 (error 108)
I20260925 00:47:04.643703 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 24732
I20260925 00:47:04.643888 24523 catalog_manager.cc:4841] TS b16d7f7338f5477ba29ab1b58f3bc36e (127.6.72.66:33245): tablet 12537c14edc6424f8c36465be4fa8600 (table test-table [id=128ec2c7902544d381120424f043bd4c]) successfully deleted
I20260925 00:47:04.644098 24521 catalog_manager.cc:4841] TS 30a0328992664a46b2a0f294cc326cab (127.6.72.67:39395): tablet 12537c14edc6424f8c36465be4fa8600 (table test-table [id=128ec2c7902544d381120424f043bd4c]) successfully deleted
I20260925 00:47:04.645560 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.645193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:41519 (local address 127.0.0.1:49430)
0925 00:47:04.645370 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:04.645452 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:41519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260925 00:47:04.645820 24521 catalog_manager.cc:4584] TS a9da5c2b05fe4d25a118251822163f14 (127.6.72.65:41519): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 12537c14edc6424f8c36465be4fa8600: Network error: Client connection negotiation failed: client connection to 127.6.72.65:41519: connect: Connection refused (error 111)
I20260925 00:47:04.650290 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 24866
I20260925 00:47:04.655773 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 24504
2026-09-25T00:47:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2342 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T00:47:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:04Z Disabled control of system clock
I20260925 00:47:04.686213 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:45207
--webserver_interface=127.6.72.126
--webserver_port=0
--builtin_ntp_servers=127.6.72.84:35147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:45207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 00:47:04.795606 25042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:04.795874 25042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:04.795912 25042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:04.799836 25042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:04.799893 25042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:04.799908 25042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:04.799925 25042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:04.799940 25042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:04.804787 25042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:45207
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:45207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:04.805845 25042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:04.807168 25042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:04.815421 25047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:04.815439 25048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:04.815649 25042 server_base.cc:1034] running on GCE node
W20260925 00:47:04.815652 25050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:04.816299 25042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:04.817350 25042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:04.818508 25042 hybrid_clock.cc:648] HybridClock initialized: now 1790297224818480 us; error 41 us; skew 500 ppm
I20260925 00:47:04.820396 25042 webserver.cc:460] Webserver started at http://127.6.72.126:35529/ using document root <none> and password file <none>
I20260925 00:47:04.820989 25042 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:04.821053 25042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:04.821296 25042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:04.823298 25042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "9d3fe9e31bc042e8870a974b85815102"
format_stamp: "Formatted at 2026-09-25 00:47:04 on dist-test-slave-w8nh"
server_key: "334999cc731f3eb670723bafaa208ff8"
server_key_iv: "1cef0be559976fec1621ae02221a9c8a"
server_key_version: "encryptionkey@0"
I20260925 00:47:04.823802 25042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "9d3fe9e31bc042e8870a974b85815102"
format_stamp: "Formatted at 2026-09-25 00:47:04 on dist-test-slave-w8nh"
server_key: "334999cc731f3eb670723bafaa208ff8"
server_key_iv: "1cef0be559976fec1621ae02221a9c8a"
server_key_version: "encryptionkey@0"
I20260925 00:47:04.827280 25042 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260925 00:47:04.829612 25056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:04.830813 25042 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:47:04.830955 25042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "9d3fe9e31bc042e8870a974b85815102"
format_stamp: "Formatted at 2026-09-25 00:47:04 on dist-test-slave-w8nh"
server_key: "334999cc731f3eb670723bafaa208ff8"
server_key_iv: "1cef0be559976fec1621ae02221a9c8a"
server_key_version: "encryptionkey@0"
I20260925 00:47:04.831063 25042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:04.844969 25042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:04.845656 25042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:04.845857 25042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:04.852779 25042 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:45207
I20260925 00:47:04.852799 25108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:45207 every 8 connection(s)
I20260925 00:47:04.853739 25042 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:04.856896 25109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:04.862157 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 25042
I20260925 00:47:04.862313 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:04.862520 25109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102: Bootstrap starting.
I20260925 00:47:04.862576 6433 external_mini_cluster.cc:1428] Setting key 1963b3e65935149c5a581185800aa5d2
I20260925 00:47:04.865164 25109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:04.865936 25109 log.cc:824] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:04.867806 25109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102: No bootstrap required, opened a new log
I20260925 00:47:04.871486 25109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3fe9e31bc042e8870a974b85815102" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45207 } }
I20260925 00:47:04.871737 25109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:04.871778 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:04.863983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50578 (local address 127.6.72.126:45207)
0925 00:47:04.864678 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:04.864694 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:04.864733 (+ 39us) server_negotiation.cc:407] Connection header received
0925 00:47:04.865434 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:04.865453 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:04.865849 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:04.866180 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:04.866603 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.867550 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.868552 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:04.869015 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:04.869204 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:04.870046 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:04.870052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:04.870065 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:04.870457 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:04.870513 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:04.870533 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:04.870638 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:04.870772 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 00:47:04.871028 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":159,"threads_started":1}
I20260925 00:47:04.871829 25109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3fe9e31bc042e8870a974b85815102, State: Initialized, Role: FOLLOWER
I20260925 00:47:04.872385 25109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3fe9e31bc042e8870a974b85815102" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45207 } }
I20260925 00:47:04.872546 25109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:04.872608 25109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:04.872697 25109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:04.873616 25109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3fe9e31bc042e8870a974b85815102" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45207 } }
I20260925 00:47:04.873986 25109 leader_election.cc:302] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d3fe9e31bc042e8870a974b85815102; no voters:
I20260925 00:47:04.874336 25109 leader_election.cc:288] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:04.874450 25114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:04.874717 25114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [term 1 LEADER]: Becoming Leader. State: Replica: 9d3fe9e31bc042e8870a974b85815102, State: Running, Role: LEADER
I20260925 00:47:04.875026 25114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3fe9e31bc042e8870a974b85815102" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45207 } }
I20260925 00:47:04.875536 25109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:04.876588 25116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d3fe9e31bc042e8870a974b85815102. Latest consensus state: current_term: 1 leader_uuid: "9d3fe9e31bc042e8870a974b85815102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3fe9e31bc042e8870a974b85815102" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45207 } } }
I20260925 00:47:04.876740 25116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:04.877024 25115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d3fe9e31bc042e8870a974b85815102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3fe9e31bc042e8870a974b85815102" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 45207 } } }
I20260925 00:47:04.877117 25115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:04.877179 25123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:04.880014 25123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:04.885701 25123 catalog_manager.cc:1347] Generated new cluster ID: fd81336b3a9a42f4ae2cb7f07b6bedf0
I20260925 00:47:04.885772 25123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:04.895198 25123 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:04.896276 25123 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:04.906088 25123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102: Generated new TSK 0
I20260925 00:47:04.906821 25123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:04.914790 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:45207
--builtin_ntp_servers=127.6.72.84:35147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 00:47:05.032684 25133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:05.032944 25133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:05.032984 25133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:05.037176 25133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:05.037245 25133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:05.037307 25133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:47:05.042223 25133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:45207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:05.043300 25133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:05.044762 25133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:05.054697 25138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:05.054857 25139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:05.054934 25133 server_base.cc:1034] running on GCE node
W20260925 00:47:05.054738 25141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:05.055672 25133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:05.056445 25133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:05.057636 25133 hybrid_clock.cc:648] HybridClock initialized: now 1790297225057593 us; error 56 us; skew 500 ppm
I20260925 00:47:05.059579 25133 webserver.cc:460] Webserver started at http://127.6.72.65:33043/ using document root <none> and password file <none>
I20260925 00:47:05.060348 25133 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:05.060443 25133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:05.060679 25133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:05.062798 25133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "775c4114a9ee4833960ce69d47c7161a"
format_stamp: "Formatted at 2026-09-25 00:47:05 on dist-test-slave-w8nh"
server_key: "0bbba34b38c39be8099e3cb71c811418"
server_key_iv: "8ed85d8445211c6df2bf6e6f62d78230"
server_key_version: "encryptionkey@0"
I20260925 00:47:05.063402 25133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "775c4114a9ee4833960ce69d47c7161a"
format_stamp: "Formatted at 2026-09-25 00:47:05 on dist-test-slave-w8nh"
server_key: "0bbba34b38c39be8099e3cb71c811418"
server_key_iv: "8ed85d8445211c6df2bf6e6f62d78230"
server_key_version: "encryptionkey@0"
I20260925 00:47:05.067235 25133 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 00:47:05.069788 25147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:05.071069 25133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:47:05.071223 25133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "775c4114a9ee4833960ce69d47c7161a"
format_stamp: "Formatted at 2026-09-25 00:47:05 on dist-test-slave-w8nh"
server_key: "0bbba34b38c39be8099e3cb71c811418"
server_key_iv: "8ed85d8445211c6df2bf6e6f62d78230"
server_key_version: "encryptionkey@0"
I20260925 00:47:05.071348 25133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:05.090782 25133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:05.091588 25133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:05.091804 25133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:05.092485 25133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:05.093443 25133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:05.093515 25133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:05.093591 25133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:05.093634 25133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:05.103228 25133 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:38269
I20260925 00:47:05.103253 25260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:38269 every 8 connection(s)
I20260925 00:47:05.104183 25133 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:47:05.111634 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 25133
I20260925 00:47:05.111790 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:47:05.112134 6433 external_mini_cluster.cc:1428] Setting key 2191896112e9b1c223b4169d36ab3e32
I20260925 00:47:05.114231 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.106272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:53901 (local address 127.6.72.126:45207)
0925 00:47:05.106423 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:05.106428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:05.107592 (+ 1164us) server_negotiation.cc:407] Connection header received
0925 00:47:05.108965 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:05.108968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:05.109019 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:05.109102 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:05.110002 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.110600 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.111834 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.112174 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.112321 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.113630 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:05.113635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:05.113638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:05.113689 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:05.113714 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:05.113727 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:05.113817 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:05.113981 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 00:47:05.114046 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 00:47:05.114410 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:45207
--builtin_ntp_servers=127.6.72.84:35147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 00:47:05.115151 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.106733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:45207 (local address 127.6.72.65:53901)
0925 00:47:05.107424 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:05.107465 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:05.108763 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:05.109282 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:05.109288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:05.109376 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:05.109866 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.109878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.110743 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.110746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.111649 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.111658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.112367 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.112371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.112514 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.113268 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:47:05.113269 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:05.113311 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:47:05.113379 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:05.113829 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:05.113837 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:05.113842 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 00:47:05.113948 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 00:47:05.114190 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":122,"threads_started":1}
I20260925 00:47:05.116624 25261 heartbeater.cc:347] Connected to a master server at 127.6.72.126:45207
I20260925 00:47:05.116880 25261 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:05.117417 25261 heartbeater.cc:511] Master 127.6.72.126:45207 requested a full tablet report, sending...
I20260925 00:47:05.118983 25073 ts_manager.cc:194] Registered new tserver with Master: 775c4114a9ee4833960ce69d47c7161a (127.6.72.65:38269)
I20260925 00:47:05.120162 25073 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.65:53901
W20260925 00:47:05.229526 25264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:05.229856 25264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:05.229928 25264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:05.234237 25264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:05.234328 25264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:05.234427 25264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:47:05.239959 25264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:45207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:05.241353 25264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:05.242801 25264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:05.253209 25269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:05.253235 25270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:05.253235 25272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:05.253307 25264 server_base.cc:1034] running on GCE node
I20260925 00:47:05.253958 25264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:05.254709 25264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:05.255924 25264 hybrid_clock.cc:648] HybridClock initialized: now 1790297225255907 us; error 30 us; skew 500 ppm
I20260925 00:47:05.258091 25264 webserver.cc:460] Webserver started at http://127.6.72.66:35617/ using document root <none> and password file <none>
I20260925 00:47:05.258744 25264 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:05.258848 25264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:05.259105 25264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:05.261652 25264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "44188ef84f5041c38565bd142f615c37"
format_stamp: "Formatted at 2026-09-25 00:47:05 on dist-test-slave-w8nh"
server_key: "400af5e15938d18f06816af10e1322b1"
server_key_iv: "1c6c05b170f826b685c55cce65405994"
server_key_version: "encryptionkey@0"
I20260925 00:47:05.262279 25264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "44188ef84f5041c38565bd142f615c37"
format_stamp: "Formatted at 2026-09-25 00:47:05 on dist-test-slave-w8nh"
server_key: "400af5e15938d18f06816af10e1322b1"
server_key_iv: "1c6c05b170f826b685c55cce65405994"
server_key_version: "encryptionkey@0"
I20260925 00:47:05.266505 25264 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:47:05.269134 25278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:05.270519 25264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 00:47:05.270668 25264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "44188ef84f5041c38565bd142f615c37"
format_stamp: "Formatted at 2026-09-25 00:47:05 on dist-test-slave-w8nh"
server_key: "400af5e15938d18f06816af10e1322b1"
server_key_iv: "1c6c05b170f826b685c55cce65405994"
server_key_version: "encryptionkey@0"
I20260925 00:47:05.270802 25264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:05.286288 25264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:05.287045 25264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:05.287269 25264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:05.287904 25264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:05.288982 25264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:05.289057 25264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:05.289139 25264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:05.289191 25264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:05.298676 25264 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:45025
I20260925 00:47:05.298704 25391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:45025 every 8 connection(s)
I20260925 00:47:05.299643 25264 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:47:05.301635 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 25264
I20260925 00:47:05.301839 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:47:05.302167 6433 external_mini_cluster.cc:1428] Setting key 6a20dfcb7312fba52cab40db2439089b
I20260925 00:47:05.304098 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:45207
--builtin_ntp_servers=127.6.72.84:35147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 00:47:05.309286 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.301596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:60923 (local address 127.6.72.126:45207)
0925 00:47:05.301789 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:05.301796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:05.302952 (+ 1156us) server_negotiation.cc:407] Connection header received
0925 00:47:05.304421 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:05.304427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:05.304491 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:05.304592 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:05.305578 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.306110 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.307273 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.307501 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.307615 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.308780 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:05.308784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:05.308787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:05.308801 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:05.308827 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:05.308835 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:05.308911 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:05.309070 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 00:47:05.309131 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 00:47:05.310148 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.302034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:45207 (local address 127.6.72.66:60923)
0925 00:47:05.302816 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:05.302870 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:05.304121 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:05.304799 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:05.304806 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:05.304892 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:05.305409 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.305423 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.306264 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.306268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.307138 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.307147 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.307655 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.307659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.307828 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.308499 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:47:05.308501 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:05.308541 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:47:05.308606 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:05.308925 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:05.308936 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:05.308945 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 00:47:05.309070 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 00:47:05.309363 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":144,"threads_started":1}
I20260925 00:47:05.311287 25392 heartbeater.cc:347] Connected to a master server at 127.6.72.126:45207
I20260925 00:47:05.311565 25392 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:05.312101 25392 heartbeater.cc:511] Master 127.6.72.126:45207 requested a full tablet report, sending...
I20260925 00:47:05.313234 25073 ts_manager.cc:194] Registered new tserver with Master: 44188ef84f5041c38565bd142f615c37 (127.6.72.66:45025)
I20260925 00:47:05.314182 25073 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.66:60923
W20260925 00:47:05.416843 25395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:05.417121 25395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:05.417167 25395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:05.421336 25395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:05.421393 25395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:05.421449 25395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:47:05.426294 25395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:45207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:05.427364 25395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:05.428810 25395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:05.438733 25403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:05.438743 25401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:05.438712 25400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:05.439191 25395 server_base.cc:1034] running on GCE node
I20260925 00:47:05.439563 25395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:05.440234 25395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:05.441501 25395 hybrid_clock.cc:648] HybridClock initialized: now 1790297225441467 us; error 45 us; skew 500 ppm
I20260925 00:47:05.443677 25395 webserver.cc:460] Webserver started at http://127.6.72.67:39995/ using document root <none> and password file <none>
I20260925 00:47:05.444391 25395 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:05.444494 25395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:05.444757 25395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:05.447045 25395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "37b1d306e404495f8a1db67fdd260e6b"
format_stamp: "Formatted at 2026-09-25 00:47:05 on dist-test-slave-w8nh"
server_key: "1497e64d5a5795726727e75ce386bc9d"
server_key_iv: "54c954700c7cc7e8175d25fbf01eab32"
server_key_version: "encryptionkey@0"
I20260925 00:47:05.447674 25395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "37b1d306e404495f8a1db67fdd260e6b"
format_stamp: "Formatted at 2026-09-25 00:47:05 on dist-test-slave-w8nh"
server_key: "1497e64d5a5795726727e75ce386bc9d"
server_key_iv: "54c954700c7cc7e8175d25fbf01eab32"
server_key_version: "encryptionkey@0"
I20260925 00:47:05.451946 25395 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:47:05.454888 25409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:05.456310 25395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:47:05.456459 25395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "37b1d306e404495f8a1db67fdd260e6b"
format_stamp: "Formatted at 2026-09-25 00:47:05 on dist-test-slave-w8nh"
server_key: "1497e64d5a5795726727e75ce386bc9d"
server_key_iv: "54c954700c7cc7e8175d25fbf01eab32"
server_key_version: "encryptionkey@0"
I20260925 00:47:05.456591 25395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:05.487089 25395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:05.487872 25395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:05.488283 25395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:05.488991 25395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:05.490006 25395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:05.490091 25395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:05.490165 25395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:05.490201 25395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:05.498854 25395 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:39145
I20260925 00:47:05.498919 25522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:39145 every 8 connection(s)
I20260925 00:47:05.499840 25395 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:47:05.500905 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 25395
I20260925 00:47:05.501032 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:47:05.501351 6433 external_mini_cluster.cc:1428] Setting key 3ebdcc67707dbf584d0dcd76c9ac96b7
I20260925 00:47:05.509871 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.501937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:43781 (local address 127.6.72.126:45207)
0925 00:47:05.502090 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:05.502094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:05.503485 (+ 1391us) server_negotiation.cc:407] Connection header received
0925 00:47:05.504952 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:05.504955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:05.505000 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:05.505086 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:05.505975 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.506510 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.507773 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.508041 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.508154 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.509298 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:05.509309 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:05.509311 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:05.509336 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:05.509359 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:05.509368 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:05.509441 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:05.509692 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 00:47:05.509749 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 00:47:05.510767 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.502472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:45207 (local address 127.6.72.67:43781)
0925 00:47:05.503340 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:05.503399 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:05.504742 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:05.505247 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:05.505255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:05.505341 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:05.505849 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.505861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.506650 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.506653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.507631 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.507643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.508212 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.508215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.508449 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.509028 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:47:05.509030 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:05.509069 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:47:05.509134 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:05.509486 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:05.509494 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:05.509499 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 00:47:05.509654 (+ 155us) client_negotiation.cc:240] Negotiation successful
0925 00:47:05.509882 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":176,"threads_started":1}
I20260925 00:47:05.512014 25523 heartbeater.cc:347] Connected to a master server at 127.6.72.126:45207
I20260925 00:47:05.512306 25523 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:05.512857 25523 heartbeater.cc:511] Master 127.6.72.126:45207 requested a full tablet report, sending...
I20260925 00:47:05.514168 25073 ts_manager.cc:194] Registered new tserver with Master: 37b1d306e404495f8a1db67fdd260e6b (127.6.72.67:39145)
I20260925 00:47:05.514993 25073 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.67:43781
I20260925 00:47:05.515646 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:47:05.521747 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.517707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50582 (local address 127.6.72.126:45207)
0925 00:47:05.517808 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:05.517812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:05.518058 (+ 246us) server_negotiation.cc:407] Connection header received
0925 00:47:05.518219 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:05.518222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:05.518270 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:05.518355 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:05.518633 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.519137 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.519966 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.520189 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.520340 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.521237 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:05.521241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:05.521245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:05.521263 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:05.521280 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:05.521288 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:05.521420 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:05.521547 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 00:47:05.521589 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 00:47:05.525552 25072 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:47:05.528872 25072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:47:05.543498 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.538264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:39145 (local address 127.0.0.1:48156)
0925 00:47:05.538520 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:05.538534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:05.538625 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:05.539509 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:05.539512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:05.539527 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:05.539630 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.539638 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.541213 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.541217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.542226 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.542235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.543050 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.543053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.543125 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.543143 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 00:47:05.543190 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 00:47:05.543339 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":41,"thread_start_us":104,"threads_started":1}
I20260925 00:47:05.543545 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.538234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43926 (local address 127.6.72.66:45025)
0925 00:47:05.538658 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:05.538670 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:05.538696 (+ 26us) server_negotiation.cc:407] Connection header received
0925 00:47:05.538775 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:05.538782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:05.538998 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:05.539185 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:05.539489 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.540549 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.541959 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.542752 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.542861 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.543067 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:05.543253 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 00:47:05.543360 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":189,"threads_started":1}
I20260925 00:47:05.543819 25538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.538711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.6.72.67:39145)
0925 00:47:05.539036 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:05.539048 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:05.539071 (+ 23us) server_negotiation.cc:407] Connection header received
0925 00:47:05.539146 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:05.539152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:05.539359 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:05.539799 (+ 440us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:05.539948 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.541081 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.542341 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.542920 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.543315 (+ 395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.543492 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:05.543542 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 00:47:05.543651 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":99,"threads_started":1}
I20260925 00:47:05.544016 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.537704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:45025 (local address 127.0.0.1:43926)
0925 00:47:05.538013 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:05.538061 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:05.538199 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:05.539172 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:05.539178 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:05.539201 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:05.539365 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.539380 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.540632 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.540638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.541832 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.541842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.542891 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.542895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.543031 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.543077 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 00:47:05.543202 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 00:47:05.543878 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":5,"spinlock_wait_cycles":3712,"thread_start_us":114,"threads_started":1}
I20260925 00:47:05.545568 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.537521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:38269 (local address 127.0.0.1:56488)
0925 00:47:05.538013 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:05.538062 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:05.538198 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:05.540362 (+ 2164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:05.540365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:05.540384 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:05.540467 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.540472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.541700 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.541703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.542881 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.542891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.545186 (+ 2295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.545189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.545277 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.545301 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 00:47:05.545362 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 00:47:05.545434 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"spinlock_wait_cycles":2816,"thread_start_us":103,"threads_started":1}
I20260925 00:47:05.545655 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.538718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.6.72.65:38269)
0925 00:47:05.539818 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:05.539829 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:05.539852 (+ 23us) server_negotiation.cc:407] Connection header received
0925 00:47:05.539990 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:05.539996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:05.540179 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:05.540382 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:05.540595 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.541566 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.544341 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.545048 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.545173 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.545363 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:05.545411 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 00:47:05.545496 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":993,"thread_start_us":169,"threads_started":1}
I20260925 00:47:05.546877 25326 tablet_service.cc:1467] Processing CreateTablet for tablet cbfef3e6b33644b78f62d8e20acc846b (DEFAULT_TABLE table=test-table [id=db62f69199244558a9e3494771b01fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:05.546877 25457 tablet_service.cc:1467] Processing CreateTablet for tablet cbfef3e6b33644b78f62d8e20acc846b (DEFAULT_TABLE table=test-table [id=db62f69199244558a9e3494771b01fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:05.548017 25326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbfef3e6b33644b78f62d8e20acc846b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:05.548029 25457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbfef3e6b33644b78f62d8e20acc846b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:05.548482 25195 tablet_service.cc:1467] Processing CreateTablet for tablet cbfef3e6b33644b78f62d8e20acc846b (DEFAULT_TABLE table=test-table [id=db62f69199244558a9e3494771b01fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:05.549367 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbfef3e6b33644b78f62d8e20acc846b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:05.555267 25540 tablet_bootstrap.cc:492] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a: Bootstrap starting.
I20260925 00:47:05.556782 25541 tablet_bootstrap.cc:492] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: Bootstrap starting.
I20260925 00:47:05.558378 25540 tablet_bootstrap.cc:654] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:05.558563 25541 tablet_bootstrap.cc:654] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:05.559237 25541 log.cc:824] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:05.559233 25540 log.cc:824] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:05.559911 25542 tablet_bootstrap.cc:492] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: Bootstrap starting.
I20260925 00:47:05.561522 25540 tablet_bootstrap.cc:492] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a: No bootstrap required, opened a new log
I20260925 00:47:05.561751 25540 ts_tablet_manager.cc:1397] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260925 00:47:05.561875 25542 tablet_bootstrap.cc:654] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:05.562656 25542 log.cc:824] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:05.564420 25541 tablet_bootstrap.cc:492] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: No bootstrap required, opened a new log
I20260925 00:47:05.564627 25541 ts_tablet_manager.cc:1397] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260925 00:47:05.564632 25542 tablet_bootstrap.cc:492] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: No bootstrap required, opened a new log
I20260925 00:47:05.564819 25542 ts_tablet_manager.cc:1397] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 00:47:05.566382 25540 raft_consensus.cc:348] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 } } peers { permanent_uuid: "44188ef84f5041c38565bd142f615c37" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45025 } } peers { permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 } }
I20260925 00:47:05.566699 25540 raft_consensus.cc:374] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:05.566772 25540 raft_consensus.cc:729] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 775c4114a9ee4833960ce69d47c7161a, State: Initialized, Role: FOLLOWER
I20260925 00:47:05.567265 25540 consensus_queue.cc:260] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 } } peers { permanent_uuid: "44188ef84f5041c38565bd142f615c37" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45025 } } peers { permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 } }
I20260925 00:47:05.568060 25261 heartbeater.cc:503] Master 127.6.72.126:45207 was elected leader, sending a full tablet report...
I20260925 00:47:05.568176 25541 raft_consensus.cc:348] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 } } peers { permanent_uuid: "44188ef84f5041c38565bd142f615c37" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45025 } } peers { permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 } }
I20260925 00:47:05.568408 25540 ts_tablet_manager.cc:1428] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260925 00:47:05.568461 25541 raft_consensus.cc:374] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:05.568508 25541 raft_consensus.cc:729] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44188ef84f5041c38565bd142f615c37, State: Initialized, Role: FOLLOWER
I20260925 00:47:05.568923 25541 consensus_queue.cc:260] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 } } peers { permanent_uuid: "44188ef84f5041c38565bd142f615c37" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45025 } } peers { permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 } }
I20260925 00:47:05.569406 25542 raft_consensus.cc:348] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 } } peers { permanent_uuid: "44188ef84f5041c38565bd142f615c37" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45025 } } peers { permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 } }
I20260925 00:47:05.569661 25542 raft_consensus.cc:374] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:05.569725 25542 raft_consensus.cc:729] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37b1d306e404495f8a1db67fdd260e6b, State: Initialized, Role: FOLLOWER
I20260925 00:47:05.570143 25542 consensus_queue.cc:260] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 } } peers { permanent_uuid: "44188ef84f5041c38565bd142f615c37" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45025 } } peers { permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 } }
I20260925 00:47:05.570431 25541 ts_tablet_manager.cc:1428] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 00:47:05.570844 25523 heartbeater.cc:503] Master 127.6.72.126:45207 was elected leader, sending a full tablet report...
I20260925 00:47:05.570987 25392 heartbeater.cc:503] Master 127.6.72.126:45207 was elected leader, sending a full tablet report...
I20260925 00:47:05.571175 25542 ts_tablet_manager.cc:1428] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260925 00:47:05.605439 25262 tablet.cc:2369] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:47:05.736050 25547 raft_consensus.cc:482] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:47:05.736334 25547 raft_consensus.cc:504] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 } } peers { permanent_uuid: "44188ef84f5041c38565bd142f615c37" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45025 } } peers { permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 } }
I20260925 00:47:05.737505 25547 leader_election.cc:288] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 775c4114a9ee4833960ce69d47c7161a (127.6.72.65:38269), 37b1d306e404495f8a1db67fdd260e6b (127.6.72.67:39145)
I20260925 00:47:05.742224 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.737770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:38269 (local address 127.6.72.66:34137)
0925 00:47:05.737904 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:05.737922 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:05.738019 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:05.738379 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:05.738382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:05.738402 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:05.738540 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.738547 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.739574 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.739576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.740835 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.740846 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.741727 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.741730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.741856 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.741904 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 00:47:05.741983 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 00:47:05.742069 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":3}
I20260925 00:47:05.742254 25538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.738211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:46179 (local address 127.6.72.67:39145)
0925 00:47:05.738358 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:05.738363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:05.738530 (+ 167us) server_negotiation.cc:407] Connection header received
0925 00:47:05.738646 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:05.738649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:05.738695 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:05.738797 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:05.739031 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.739698 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.741097 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.741687 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.741788 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.741848 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:05.742079 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 00:47:05.742150 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 00:47:05.742260 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.737916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:34137 (local address 127.6.72.65:38269)
0925 00:47:05.738054 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:05.738060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:05.738078 (+ 18us) server_negotiation.cc:407] Connection header received
0925 00:47:05.738147 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:05.738149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:05.738195 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:05.738312 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:05.738657 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.739464 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.740972 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.741604 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.741722 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.741799 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:05.742031 (+ 232us) server_negotiation.cc:299] Negotiation successful
0925 00:47:05.742114 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:47:05.742609 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.738063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:39145 (local address 127.6.72.66:46179)
0925 00:47:05.738438 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:05.738451 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:05.738542 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:05.738819 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:05.738821 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:05.738836 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:05.738924 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.738929 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.739803 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.739806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.740987 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:05.740993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.741814 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.741818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:05.741936 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.741967 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 00:47:05.742407 (+ 440us) client_negotiation.cc:240] Negotiation successful
0925 00:47:05.742486 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":22,"thread_start_us":150,"threads_started":1}
I20260925 00:47:05.742944 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbfef3e6b33644b78f62d8e20acc846b" candidate_uuid: "44188ef84f5041c38565bd142f615c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775c4114a9ee4833960ce69d47c7161a" is_pre_election: true
I20260925 00:47:05.743208 25477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbfef3e6b33644b78f62d8e20acc846b" candidate_uuid: "44188ef84f5041c38565bd142f615c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37b1d306e404495f8a1db67fdd260e6b" is_pre_election: true
I20260925 00:47:05.743291 25215 raft_consensus.cc:2454] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44188ef84f5041c38565bd142f615c37 in term 0.
I20260925 00:47:05.743474 25477 raft_consensus.cc:2454] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44188ef84f5041c38565bd142f615c37 in term 0.
I20260925 00:47:05.743814 25279 leader_election.cc:302] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44188ef84f5041c38565bd142f615c37, 775c4114a9ee4833960ce69d47c7161a; no voters:
I20260925 00:47:05.744047 25547 raft_consensus.cc:2781] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:47:05.744133 25547 raft_consensus.cc:482] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:47:05.744164 25547 raft_consensus.cc:3037] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:05.745245 25547 raft_consensus.cc:504] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 } } peers { permanent_uuid: "44188ef84f5041c38565bd142f615c37" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45025 } } peers { permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 } }
I20260925 00:47:05.745602 25547 leader_election.cc:288] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [CANDIDATE]: Term 1 election: Requested vote from peers 775c4114a9ee4833960ce69d47c7161a (127.6.72.65:38269), 37b1d306e404495f8a1db67fdd260e6b (127.6.72.67:39145)
I20260925 00:47:05.745961 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbfef3e6b33644b78f62d8e20acc846b" candidate_uuid: "44188ef84f5041c38565bd142f615c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775c4114a9ee4833960ce69d47c7161a"
I20260925 00:47:05.746016 25477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbfef3e6b33644b78f62d8e20acc846b" candidate_uuid: "44188ef84f5041c38565bd142f615c37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37b1d306e404495f8a1db67fdd260e6b"
I20260925 00:47:05.746094 25215 raft_consensus.cc:3037] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:05.746165 25477 raft_consensus.cc:3037] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:05.747125 25477 raft_consensus.cc:2454] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44188ef84f5041c38565bd142f615c37 in term 1.
I20260925 00:47:05.747133 25215 raft_consensus.cc:2454] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44188ef84f5041c38565bd142f615c37 in term 1.
I20260925 00:47:05.747447 25279 leader_election.cc:302] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44188ef84f5041c38565bd142f615c37, 775c4114a9ee4833960ce69d47c7161a; no voters:
I20260925 00:47:05.747645 25547 raft_consensus.cc:2781] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:05.747884 25547 raft_consensus.cc:686] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 1 LEADER]: Becoming Leader. State: Replica: 44188ef84f5041c38565bd142f615c37, State: Running, Role: LEADER
I20260925 00:47:05.748281 25547 consensus_queue.cc:237] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 } } peers { permanent_uuid: "44188ef84f5041c38565bd142f615c37" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45025 } } peers { permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 } }
W20260925 00:47:05.750844 25524 tablet.cc:2369] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:47:05.751561 25071 catalog_manager.cc:5495] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44188ef84f5041c38565bd142f615c37 (127.6.72.66). New cstate: current_term: 1 leader_uuid: "44188ef84f5041c38565bd142f615c37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44188ef84f5041c38565bd142f615c37" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 45025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 } health_report { overall_health: UNKNOWN } } }
I20260925 00:47:05.783475 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:05.780312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.6.72.66:45025)
0925 00:47:05.780441 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:05.780444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:05.780458 (+ 14us) server_negotiation.cc:407] Connection header received
0925 00:47:05.780526 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:05.780528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:05.780583 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:05.780678 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:05.780893 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.781393 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.782177 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:05.782393 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:05.782470 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:05.782718 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:05.783127 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:05.783246 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:05.783326 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 00:47:05.783359 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 00:47:05.789500 25215 raft_consensus.cc:1261] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a [term 1 FOLLOWER]: Refusing update from remote peer 44188ef84f5041c38565bd142f615c37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:05.789500 25477 raft_consensus.cc:1261] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b [term 1 FOLLOWER]: Refusing update from remote peer 44188ef84f5041c38565bd142f615c37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:05.790230 25547 consensus_queue.cc:1035] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b1d306e404495f8a1db67fdd260e6b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:05.790464 25550 consensus_queue.cc:1035] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "775c4114a9ee4833960ce69d47c7161a" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:05.797508 25553 mvcc.cc:205] Tried to move back new op lower bound from 7333057436826148864 to 7333057436667633664. Current Snapshot: MvccSnapshot[applied={T|T < 7333057436826148864}]
I20260925 00:47:05.798099 25555 mvcc.cc:205] Tried to move back new op lower bound from 7333057436826148864 to 7333057436667633664. Current Snapshot: MvccSnapshot[applied={T|T < 7333057436826148864}]
I20260925 00:47:05.799546 25554 mvcc.cc:205] Tried to move back new op lower bound from 7333057436826148864 to 7333057436667633664. Current Snapshot: MvccSnapshot[applied={T|T < 7333057436826148864}]
I20260925 00:47:05.806890 25072 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:05.807099 25072 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:05.809633 25072 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d3fe9e31bc042e8870a974b85815102: Sending DeleteTablet for 3 replicas of tablet cbfef3e6b33644b78f62d8e20acc846b
I20260925 00:47:05.810458 25457 tablet_service.cc:1514] Processing DeleteTablet for tablet cbfef3e6b33644b78f62d8e20acc846b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:05 UTC) from {username='slave'} at 127.0.0.1:48156
I20260925 00:47:05.810429 25326 tablet_service.cc:1514] Processing DeleteTablet for tablet cbfef3e6b33644b78f62d8e20acc846b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:05 UTC) from {username='slave'} at 127.0.0.1:43926
I20260925 00:47:05.810954 25563 tablet_replica.cc:330] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: stopping tablet replica
I20260925 00:47:05.810956 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet cbfef3e6b33644b78f62d8e20acc846b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:05 UTC) from {username='slave'} at 127.0.0.1:56488
I20260925 00:47:05.811328 25563 raft_consensus.cc:2229] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:05.811482 25565 tablet_replica.cc:330] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a: stopping tablet replica
I20260925 00:47:05.811672 25563 raft_consensus.cc:2258] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:05.811878 25565 raft_consensus.cc:2229] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:05.812026 25564 tablet_replica.cc:330] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: stopping tablet replica
I20260925 00:47:05.812302 25565 raft_consensus.cc:2258] T cbfef3e6b33644b78f62d8e20acc846b P 775c4114a9ee4833960ce69d47c7161a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:05.812359 25564 raft_consensus.cc:2229] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:47:05.812395 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 25133
I20260925 00:47:05.813735 25564 raft_consensus.cc:2258] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:05.815223 25564 ts_tablet_manager.cc:1905] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:05.817342 25563 ts_tablet_manager.cc:1905] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:05.818681 25564 ts_tablet_manager.cc:1918] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:05.818785 25564 log.cc:1196] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/cbfef3e6b33644b78f62d8e20acc846b
I20260925 00:47:05.819114 25564 ts_tablet_manager.cc:1939] T cbfef3e6b33644b78f62d8e20acc846b P 44188ef84f5041c38565bd142f615c37: Deleting consensus metadata
I20260925 00:47:05.820116 25563 ts_tablet_manager.cc:1918] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:05.820330 25563 log.cc:1196] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/cbfef3e6b33644b78f62d8e20acc846b
I20260925 00:47:05.820438 25060 catalog_manager.cc:4841] TS 44188ef84f5041c38565bd142f615c37 (127.6.72.66:45025): tablet cbfef3e6b33644b78f62d8e20acc846b (table test-table [id=db62f69199244558a9e3494771b01fb9]) successfully deleted
I20260925 00:47:05.820755 25563 ts_tablet_manager.cc:1939] T cbfef3e6b33644b78f62d8e20acc846b P 37b1d306e404495f8a1db67fdd260e6b: Deleting consensus metadata
W20260925 00:47:05.821584 25057 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:38269 (error 108)
I20260925 00:47:05.822079 25057 catalog_manager.cc:4841] TS 37b1d306e404495f8a1db67fdd260e6b (127.6.72.67:39145): tablet cbfef3e6b33644b78f62d8e20acc846b (table test-table [id=db62f69199244558a9e3494771b01fb9]) successfully deleted
W20260925 00:47:05.822690 25534 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:05.822380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:38269 (local address 127.0.0.1:56500)
0925 00:47:05.822520 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:05.822594 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:38269: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 00:47:05.822850 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 25264
W20260925 00:47:05.822861 25057 catalog_manager.cc:4584] TS 775c4114a9ee4833960ce69d47c7161a (127.6.72.65:38269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cbfef3e6b33644b78f62d8e20acc846b: Network error: Client connection negotiation failed: client connection to 127.6.72.65:38269: connect: Connection reset by peer (error 104)
I20260925 00:47:05.828754 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 25395
I20260925 00:47:05.834645 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 25042
2026-09-25T00:47:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1178 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:05 dist-test-slave-w8nh krb5kdc[25569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:05 dist-test-slave-w8nh krb5kdc[25569](info): set up 2 sockets
Sep 25 00:47:05 dist-test-slave-w8nh krb5kdc[25569](info): commencing operation
krb5kdc: starting...
W20260925 00:47:06.888387 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:06 dist-test-slave-w8nh krb5kdc[25569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:47:07.015941 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:46205
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:46205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:07.128949 25582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:07.129387 25582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:07.129482 25582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:07.136475 25582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:07.136550 25582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:07.136570 25582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:07.136590 25582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:07.136641 25582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:07.142459 25582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:46205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:46205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:07.143643 25582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:07.144799 25582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:07.153538 25588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:07.153550 25587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:07.153831 25582 server_base.cc:1034] running on GCE node
W20260925 00:47:07.153550 25590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:07.154726 25582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:07.155975 25582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:07.157191 25582 hybrid_clock.cc:648] HybridClock initialized: now 1790297227157187 us; error 59 us; skew 500 ppm
Sep 25 00:47:07 dist-test-slave-w8nh krb5kdc[25569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297227, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:07.160135 25582 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:47:07.161118 25582 webserver.cc:460] Webserver started at http://127.6.72.126:42831/ using document root <none> and password file <none>
I20260925 00:47:07.161751 25582 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:07.161850 25582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:07.162096 25582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:07.164207 25582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "fb98a44a50344917bbba8dff75c129de"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "5e49dc1ef0a95e39f59d7be87fe83bbf"
server_key_iv: "db0191769dbae9bcfa222fd673b37623"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.164875 25582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "fb98a44a50344917bbba8dff75c129de"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "5e49dc1ef0a95e39f59d7be87fe83bbf"
server_key_iv: "db0191769dbae9bcfa222fd673b37623"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.168768 25582 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:47:07.171306 25597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:07.172673 25582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:07.172820 25582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "fb98a44a50344917bbba8dff75c129de"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "5e49dc1ef0a95e39f59d7be87fe83bbf"
server_key_iv: "db0191769dbae9bcfa222fd673b37623"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.172945 25582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:07.186231 25582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:07.189437 25582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:07.189661 25582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:07.197379 25582 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:46205
I20260925 00:47:07.197393 25649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:46205 every 8 connection(s)
I20260925 00:47:07.198422 25582 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:07.201418 25650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:07.202230 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 25582
I20260925 00:47:07.202376 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:07.202695 6433 external_mini_cluster.cc:1428] Setting key 7463f634da837413dfb751c255c21195
I20260925 00:47:07.207924 25650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de: Bootstrap starting.
I20260925 00:47:07.210433 25650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:07.211208 25650 log.cc:824] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de: Log is configured to *not* fsync() on all Append() calls
Sep 25 00:47:07 dist-test-slave-w8nh krb5kdc[25569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:07.213394 25650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de: No bootstrap required, opened a new log
I20260925 00:47:07.217319 25650 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb98a44a50344917bbba8dff75c129de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46205 } }
I20260925 00:47:07.217535 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:07.204217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.6.72.126:46205)
0925 00:47:07.204708 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:07.204732 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:07.204769 (+ 37us) server_negotiation.cc:407] Connection header received
0925 00:47:07.205547 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:07.205566 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:07.205924 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:07.206271 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:07.207119 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.208165 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.209060 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.209551 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.209735 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:07.212667 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:07.212672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:07.212687 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:07.212715 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:07.214528 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:07.215023 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:07.215028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:07.215032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:07.215145 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:07.215493 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:07.215496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:07.215498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:07.215913 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:07.216124 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:07.216473 (+ 349us) server_negotiation.cc:299] Negotiation successful
0925 00:47:07.216779 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":140,"threads_started":1}
I20260925 00:47:07.217576 25650 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:07.217640 25650 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb98a44a50344917bbba8dff75c129de, State: Initialized, Role: FOLLOWER
I20260925 00:47:07.218088 25650 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb98a44a50344917bbba8dff75c129de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46205 } }
I20260925 00:47:07.218243 25650 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:07.218331 25650 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:07.218436 25650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:07.219688 25650 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb98a44a50344917bbba8dff75c129de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46205 } }
I20260925 00:47:07.220089 25650 leader_election.cc:302] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb98a44a50344917bbba8dff75c129de; no voters:
I20260925 00:47:07.220508 25650 leader_election.cc:288] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:07.220618 25655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:07.220862 25655 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [term 1 LEADER]: Becoming Leader. State: Replica: fb98a44a50344917bbba8dff75c129de, State: Running, Role: LEADER
I20260925 00:47:07.221222 25655 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb98a44a50344917bbba8dff75c129de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46205 } }
I20260925 00:47:07.221776 25650 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:07.223097 25656 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb98a44a50344917bbba8dff75c129de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb98a44a50344917bbba8dff75c129de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46205 } } }
I20260925 00:47:07.223234 25656 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:07.223618 25657 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb98a44a50344917bbba8dff75c129de. Latest consensus state: current_term: 1 leader_uuid: "fb98a44a50344917bbba8dff75c129de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb98a44a50344917bbba8dff75c129de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 46205 } } }
I20260925 00:47:07.223759 25657 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de [sys.catalog]: This master's current role is: LEADER
W20260925 00:47:07.226666 25670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:07.226775 25670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:07.226864 25664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:07.227739 25664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:07.234139 25664 catalog_manager.cc:1347] Generated new cluster ID: f95f2c79d0a143c6be49752d214ccddf
I20260925 00:47:07.234236 25664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:07.267004 25664 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:07.268006 25664 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:07.277422 25664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de: Generated new TSK 0
I20260925 00:47:07.278203 25664 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.72.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
Entry for principal kudu/127.6.72.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab.
I20260925 00:47:07.315601 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46205
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:07.430112 25676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:07.430388 25676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:07.430433 25676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:07.434480 25676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:07.434541 25676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:07.434641 25676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:47:07.440416 25676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.65.keytab
--principal=kudu/127.6.72.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:07.441558 25676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:07.442667 25676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:07.452508 25682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:07.452526 25684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:07.452581 25676 server_base.cc:1034] running on GCE node
W20260925 00:47:07.452483 25681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:07.453155 25676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:07.453851 25676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:07.455088 25676 hybrid_clock.cc:648] HybridClock initialized: now 1790297227455067 us; error 32 us; skew 500 ppm
Sep 25 00:47:07 dist-test-slave-w8nh krb5kdc[25569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297227, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:07.457985 25676 init.cc:377] Logged in from keytab as kudu/127.6.72.65@KRBTEST.COM (short username kudu)
I20260925 00:47:07.459005 25676 webserver.cc:460] Webserver started at http://127.6.72.65:40525/ using document root <none> and password file <none>
I20260925 00:47:07.459685 25676 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:07.459790 25676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:07.460045 25676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:07.462440 25676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "7865e662099d43be9efe6a9ec9d8b98f"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "15df15b427bb38a78084cfbf8aa12e91"
server_key_iv: "3bd0f35a1c350251978826d571aefdd7"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.463057 25676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "7865e662099d43be9efe6a9ec9d8b98f"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "15df15b427bb38a78084cfbf8aa12e91"
server_key_iv: "3bd0f35a1c350251978826d571aefdd7"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.467159 25676 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260925 00:47:07.469834 25691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:07.471125 25676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:47:07.471278 25676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "7865e662099d43be9efe6a9ec9d8b98f"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "15df15b427bb38a78084cfbf8aa12e91"
server_key_iv: "3bd0f35a1c350251978826d571aefdd7"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.471403 25676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:07.483978 25676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:07.487401 25676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:07.487558 25676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:07.488123 25676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:07.489001 25676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:07.489051 25676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:07.489094 25676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:07.489126 25676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:07.498459 25676 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:41833
I20260925 00:47:07.498483 25804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:41833 every 8 connection(s)
I20260925 00:47:07.499397 25676 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:47:07.502784 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 25676
I20260925 00:47:07.502933 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:47:07.503291 6433 external_mini_cluster.cc:1428] Setting key 3ff53f9e0d91128daaaee595a08b04bb
Sep 25 00:47:07 dist-test-slave-w8nh krb5kdc[25569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297227, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.65@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.66@KRBTEST.COM; defaulting to no policy
I20260925 00:47:07.514989 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:07.501305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:55967 (local address 127.6.72.126:46205)
0925 00:47:07.501460 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:07.501465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:07.502359 (+ 894us) server_negotiation.cc:407] Connection header received
0925 00:47:07.503592 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:07.503595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:07.503642 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:07.503730 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:07.505371 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.505909 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.507457 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.507833 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.507922 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:07.510939 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:07.510946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:07.510953 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:07.510994 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:07.512795 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:07.513300 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:07.513305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:07.513307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:07.513401 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:07.513743 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:07.513749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:07.513751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:07.514057 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:07.514185 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:07.514679 (+ 494us) server_negotiation.cc:299] Negotiation successful
0925 00:47:07.514846 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 00:47:07.515919 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:07.501715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:46205 (local address 127.6.72.65:55967)
0925 00:47:07.502192 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:07.502238 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:07.503356 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:07.503936 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:07.503947 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:07.504482 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:07.505203 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:07.505228 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.506046 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.506054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:07.507321 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:07.507331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.508003 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.508010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:07.508318 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:07.508985 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:07.509013 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:07.510757 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:07.512918 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:07.512923 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:07.512936 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:07.513191 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:07.513512 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:07.513514 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:07.513516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:07.513644 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:07.514167 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:07.514173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:07.514407 (+ 234us) client_negotiation.cc:769] Sending connection context
0925 00:47:07.514612 (+ 205us) client_negotiation.cc:240] Negotiation successful
0925 00:47:07.514896 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":115,"threads_started":1}
I20260925 00:47:07.517431 25805 heartbeater.cc:347] Connected to a master server at 127.6.72.126:46205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.66@KRBTEST.COM" created.
I20260925 00:47:07.517684 25805 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:07.518314 25805 heartbeater.cc:511] Master 127.6.72.126:46205 requested a full tablet report, sending...
I20260925 00:47:07.519917 25614 ts_manager.cc:194] Registered new tserver with Master: 7865e662099d43be9efe6a9ec9d8b98f (127.6.72.65:41833)
I20260925 00:47:07.521780 25614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.65@KRBTEST.COM'} at 127.6.72.65:55967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
Entry for principal kudu/127.6.72.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab.
I20260925 00:47:07.535418 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46205
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:07.651505 25810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:07.651791 25810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:07.651836 25810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:07.656004 25810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:07.656078 25810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:07.656140 25810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:47:07.661659 25810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.66.keytab
--principal=kudu/127.6.72.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:07.662675 25810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:07.663661 25810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:07.673787 25816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:07.673965 25810 server_base.cc:1034] running on GCE node
W20260925 00:47:07.673832 25818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:07.673832 25815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:07.674578 25810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:07.675421 25810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:07.676602 25810 hybrid_clock.cc:648] HybridClock initialized: now 1790297227676585 us; error 33 us; skew 500 ppm
Sep 25 00:47:07 dist-test-slave-w8nh krb5kdc[25569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297227, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:07.679824 25810 init.cc:377] Logged in from keytab as kudu/127.6.72.66@KRBTEST.COM (short username kudu)
I20260925 00:47:07.680876 25810 webserver.cc:460] Webserver started at http://127.6.72.66:38099/ using document root <none> and password file <none>
I20260925 00:47:07.681473 25810 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:07.681567 25810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:07.681814 25810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:07.683913 25810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "9d08340d3d79c837f73bef8570737396"
server_key_iv: "dadd96ea827b578d96a93efc095e19af"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.684577 25810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "9d08340d3d79c837f73bef8570737396"
server_key_iv: "dadd96ea827b578d96a93efc095e19af"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.688580 25810 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 00:47:07.691048 25825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:07.692415 25810 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:47:07.692559 25810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "9d08340d3d79c837f73bef8570737396"
server_key_iv: "dadd96ea827b578d96a93efc095e19af"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.692682 25810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:07.714411 25810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:07.717877 25810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:07.718097 25810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:07.718720 25810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:07.719588 25810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:07.719660 25810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:07.719734 25810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:07.719787 25810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:07.729307 25810 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:46399
I20260925 00:47:07.729331 25938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:46399 every 8 connection(s)
I20260925 00:47:07.730238 25810 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:47:07.733047 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 25810
I20260925 00:47:07.733223 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:47:07.733548 6433 external_mini_cluster.cc:1428] Setting key b7221e271753e21ddd11c5af5a5959bc
Sep 25 00:47:07 dist-test-slave-w8nh krb5kdc[25569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297227, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.66@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.72.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.67@KRBTEST.COM" created.
I20260925 00:47:07.746294 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:07.732216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:33725 (local address 127.6.72.126:46205)
0925 00:47:07.732449 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:07.732454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:07.733715 (+ 1261us) server_negotiation.cc:407] Connection header received
0925 00:47:07.734836 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:07.734840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:07.734900 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:07.734979 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:07.736192 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.737038 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.738913 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.739132 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.739224 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:07.742370 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:07.742373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:07.742376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:07.742413 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:07.743901 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:07.744557 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:07.744561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:07.744563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:07.744648 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:07.745030 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:07.745033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:07.745035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:07.745230 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:07.745409 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:07.745989 (+ 580us) server_negotiation.cc:299] Negotiation successful
0925 00:47:07.746102 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260925 00:47:07.747493 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:07.732706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:46205 (local address 127.6.72.66:33725)
0925 00:47:07.733525 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:07.733597 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:07.734587 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:07.735182 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:07.735190 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:07.735557 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:07.736039 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:07.736051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.737171 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.737176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:07.738687 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:07.738701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.739311 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.739317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:07.739583 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:07.740273 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:07.740317 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:07.742158 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:07.744036 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:07.744042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:07.744062 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:07.744461 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:07.744767 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:07.744770 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:07.744772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:07.744922 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:07.745411 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:07.745417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:07.745686 (+ 269us) client_negotiation.cc:769] Sending connection context
0925 00:47:07.745874 (+ 188us) client_negotiation.cc:240] Negotiation successful
0925 00:47:07.746296 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":126,"threads_started":1}
I20260925 00:47:07.749028 25939 heartbeater.cc:347] Connected to a master server at 127.6.72.126:46205
I20260925 00:47:07.749317 25939 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:07.749902 25939 heartbeater.cc:511] Master 127.6.72.126:46205 requested a full tablet report, sending...
I20260925 00:47:07.751144 25614 ts_manager.cc:194] Registered new tserver with Master: 55cb87a0ef374a73a9fcb39b4e5fda4a (127.6.72.66:46399)
I20260925 00:47:07.751839 25614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.66@KRBTEST.COM'} at 127.6.72.66:33725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
Entry for principal kudu/127.6.72.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab.
I20260925 00:47:07.764552 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46205
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:39989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:07.878625 25944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:07.878924 25944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:07.879007 25944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:07.883320 25944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:07.883404 25944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:07.883468 25944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:47:07.889550 25944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:39989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.67.keytab
--principal=kudu/127.6.72.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:46205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:07.890705 25944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:07.891688 25944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:07.902520 25949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:07.902490 25952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:07.902495 25950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:07.902776 25944 server_base.cc:1034] running on GCE node
I20260925 00:47:07.903172 25944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:07.903820 25944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:07.904989 25944 hybrid_clock.cc:648] HybridClock initialized: now 1790297227904967 us; error 33 us; skew 500 ppm
Sep 25 00:47:07 dist-test-slave-w8nh krb5kdc[25569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297227, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:07.907977 25944 init.cc:377] Logged in from keytab as kudu/127.6.72.67@KRBTEST.COM (short username kudu)
I20260925 00:47:07.908965 25944 webserver.cc:460] Webserver started at http://127.6.72.67:44881/ using document root <none> and password file <none>
I20260925 00:47:07.909565 25944 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:07.909659 25944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:07.909904 25944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:07.912043 25944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "571116e9c7384dc89a22b00acde5b9b0"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "7a4c68c992b5218566adc26c32f6cabd"
server_key_iv: "d73c0abd34cdcbecefa3e4266ded9426"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.912678 25944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "571116e9c7384dc89a22b00acde5b9b0"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "7a4c68c992b5218566adc26c32f6cabd"
server_key_iv: "d73c0abd34cdcbecefa3e4266ded9426"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.916762 25944 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:47:07.919561 25959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:07.920982 25944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:07.921125 25944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "571116e9c7384dc89a22b00acde5b9b0"
format_stamp: "Formatted at 2026-09-25 00:47:07 on dist-test-slave-w8nh"
server_key: "7a4c68c992b5218566adc26c32f6cabd"
server_key_iv: "d73c0abd34cdcbecefa3e4266ded9426"
server_key_version: "encryptionkey@0"
I20260925 00:47:07.921252 25944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:07.937925 25944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:07.941440 25944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:07.941654 25944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:07.942262 25944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:07.943099 25944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:07.943176 25944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:07.943272 25944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:07.943323 25944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:07.952518 25944 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:36203
I20260925 00:47:07.952536 26072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:36203 every 8 connection(s)
I20260925 00:47:07.953419 25944 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:47:07.961621 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 25944
I20260925 00:47:07.961774 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:47:07.962141 6433 external_mini_cluster.cc:1428] Setting key 506642e3b89f0baf4c87e84618dce097
Sep 25 00:47:07 dist-test-slave-w8nh krb5kdc[25569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297227, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.67@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:07.967471 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:07.955252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:41939 (local address 127.6.72.126:46205)
0925 00:47:07.955412 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:07.955417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:07.956622 (+ 1205us) server_negotiation.cc:407] Connection header received
0925 00:47:07.957535 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:07.957538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:07.957587 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:07.957665 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:07.958836 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.959365 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.960590 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.960827 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.960914 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:07.963832 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:07.963835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:07.963838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:07.963873 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:07.965511 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:07.966010 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:07.966013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:07.966014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:07.966090 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:07.966408 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:07.966411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:07.966412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:07.966592 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:07.966678 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:07.967222 (+ 544us) server_negotiation.cc:299] Negotiation successful
0925 00:47:07.967346 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 00:47:07.968351 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:07.955691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:46205 (local address 127.6.72.67:41939)
0925 00:47:07.956427 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:07.956470 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:07.957342 (+ 872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:07.957826 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:07.957832 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:07.958211 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:07.958704 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:07.958717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.959520 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.959523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:07.960469 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:07.960480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.960973 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.960976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:07.961120 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:07.961761 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:07.961784 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:07.963672 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:07.965616 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:07.965621 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:07.965635 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:07.965916 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:07.966189 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:07.966191 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:07.966192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:07.966330 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:07.966683 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:07.966688 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:07.966955 (+ 267us) client_negotiation.cc:769] Sending connection context
0925 00:47:07.967122 (+ 167us) client_negotiation.cc:240] Negotiation successful
0925 00:47:07.967397 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":119,"threads_started":1}
I20260925 00:47:07.969568 26073 heartbeater.cc:347] Connected to a master server at 127.6.72.126:46205
I20260925 00:47:07.969882 26073 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:07.970415 26073 heartbeater.cc:511] Master 127.6.72.126:46205 requested a full tablet report, sending...
I20260925 00:47:07.971483 25614 ts_manager.cc:194] Registered new tserver with Master: 571116e9c7384dc89a22b00acde5b9b0 (127.6.72.67:36203)
I20260925 00:47:07.972070 25614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.72.67@KRBTEST.COM'} at 127.6.72.67:41939
I20260925 00:47:07.976507 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:47:07.985754 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:07.978576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51056 (local address 127.6.72.126:46205)
0925 00:47:07.978705 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:07.978709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:07.978885 (+ 176us) server_negotiation.cc:407] Connection header received
0925 00:47:07.978971 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:07.978973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:07.979012 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:07.979077 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:07.979774 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.980343 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.981161 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:07.981439 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:07.981516 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:07.983028 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:07.983030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:07.983032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:07.983049 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:07.984318 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:07.984702 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:07.984705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:07.984706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:07.984779 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:07.984994 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:07.984996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:07.984997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:07.985205 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:07.985312 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:07.985535 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 00:47:07.985632 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 00:47:07.988747 25614 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:47:07.991729 25614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:47:08.006266 26084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.000204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:41833 (local address 127.0.0.1:57112)
0925 00:47:08.000706 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:08.000762 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:08.000896 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:08.001738 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:08.001743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:08.001762 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:08.001867 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.001871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.003277 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.003280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.004745 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.004756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.005698 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.005702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.005806 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.005842 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 00:47:08.005979 (+ 137us) client_negotiation.cc:240] Negotiation successful
0925 00:47:08.006106 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":63,"thread_start_us":104,"threads_started":1}
I20260925 00:47:08.006276 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.000396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57112 (local address 127.6.72.65:41833)
0925 00:47:08.000822 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:08.000840 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:08.000868 (+ 28us) server_negotiation.cc:407] Connection header received
0925 00:47:08.001151 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:08.001160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:08.001377 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:08.001527 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:08.002026 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.003134 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.004880 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.005573 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.005699 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.005851 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:08.006009 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 00:47:08.006112 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":76,"threads_started":1}
I20260925 00:47:08.006713 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.000986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:36203 (local address 127.0.0.1:55866)
0925 00:47:08.001557 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:08.001571 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:08.001663 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:08.002592 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:08.002595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:08.002607 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:08.002725 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.002733 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.003990 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.003993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.005064 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.005071 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.006166 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.006170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.006256 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.006278 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 00:47:08.006474 (+ 196us) client_negotiation.cc:240] Negotiation successful
0925 00:47:08.006561 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":16,"thread_start_us":92,"threads_started":1}
I20260925 00:47:08.006866 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.001705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44174 (local address 127.6.72.66:46399)
0925 00:47:08.002077 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:08.002089 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:08.002112 (+ 23us) server_negotiation.cc:407] Connection header received
0925 00:47:08.002181 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:08.002186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:08.002370 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:08.002520 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:08.002975 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.003903 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.005271 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.006026 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.006141 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.006285 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:08.006610 (+ 325us) server_negotiation.cc:299] Negotiation successful
0925 00:47:08.006720 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":147,"threads_started":1}
I20260925 00:47:08.006713 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.001328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:46399 (local address 127.0.0.1:44174)
0925 00:47:08.001894 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:08.001907 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:08.001983 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:08.002521 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:08.002527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:08.002551 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:08.002861 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.002867 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.004044 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.004047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.005136 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.005146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.006345 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.006348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.006450 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.006479 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 00:47:08.006550 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 00:47:08.006600 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":82,"thread_start_us":71,"threads_started":1}
I20260925 00:47:08.007635 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.001167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55866 (local address 127.6.72.67:36203)
0925 00:47:08.002196 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:08.002204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:08.002221 (+ 17us) server_negotiation.cc:407] Connection header received
0925 00:47:08.002288 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:08.002292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:08.002429 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:08.002690 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:08.002854 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.003840 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.005188 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.006024 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.007102 (+ 1078us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.007351 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:08.007405 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 00:47:08.007496 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":947,"thread_start_us":128,"threads_started":1}
I20260925 00:47:08.009752 25739 tablet_service.cc:1467] Processing CreateTablet for tablet 36ab0d20c49447c49fe0b9f26f3773bc (DEFAULT_TABLE table=test-table [id=aa1ad0ff32794bd69e8677e5a1622065]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:08.010047 25873 tablet_service.cc:1467] Processing CreateTablet for tablet 36ab0d20c49447c49fe0b9f26f3773bc (DEFAULT_TABLE table=test-table [id=aa1ad0ff32794bd69e8677e5a1622065]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:08.010365 26007 tablet_service.cc:1467] Processing CreateTablet for tablet 36ab0d20c49447c49fe0b9f26f3773bc (DEFAULT_TABLE table=test-table [id=aa1ad0ff32794bd69e8677e5a1622065]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:08.010937 25739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36ab0d20c49447c49fe0b9f26f3773bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:08.011032 25873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36ab0d20c49447c49fe0b9f26f3773bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:08.011288 26007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36ab0d20c49447c49fe0b9f26f3773bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:08.017894 26090 tablet_bootstrap.cc:492] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: Bootstrap starting.
I20260925 00:47:08.018745 26092 tablet_bootstrap.cc:492] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: Bootstrap starting.
I20260925 00:47:08.019127 26091 tablet_bootstrap.cc:492] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f: Bootstrap starting.
I20260925 00:47:08.020885 26090 tablet_bootstrap.cc:654] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:08.020908 26091 tablet_bootstrap.cc:654] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:08.021008 26092 tablet_bootstrap.cc:654] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:08.021786 26091 log.cc:824] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:08.021885 26090 log.cc:824] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:08.022396 26092 log.cc:824] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:08.024118 26091 tablet_bootstrap.cc:492] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f: No bootstrap required, opened a new log
I20260925 00:47:08.024212 26090 tablet_bootstrap.cc:492] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: No bootstrap required, opened a new log
I20260925 00:47:08.024310 26092 tablet_bootstrap.cc:492] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: No bootstrap required, opened a new log
I20260925 00:47:08.024369 26091 ts_tablet_manager.cc:1397] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260925 00:47:08.024432 26090 ts_tablet_manager.cc:1397] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260925 00:47:08.024451 26092 ts_tablet_manager.cc:1397] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260925 00:47:08.028143 26091 raft_consensus.cc:348] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 } } peers { permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 } } peers { permanent_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46399 } }
I20260925 00:47:08.028474 26091 raft_consensus.cc:374] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:08.028539 26091 raft_consensus.cc:729] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7865e662099d43be9efe6a9ec9d8b98f, State: Initialized, Role: FOLLOWER
I20260925 00:47:08.028378 26090 raft_consensus.cc:348] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 } } peers { permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 } } peers { permanent_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46399 } }
I20260925 00:47:08.028378 26092 raft_consensus.cc:348] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 } } peers { permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 } } peers { permanent_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46399 } }
I20260925 00:47:08.028640 26090 raft_consensus.cc:374] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:08.028651 26092 raft_consensus.cc:374] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:08.028704 26092 raft_consensus.cc:729] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571116e9c7384dc89a22b00acde5b9b0, State: Initialized, Role: FOLLOWER
I20260925 00:47:08.028705 26090 raft_consensus.cc:729] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55cb87a0ef374a73a9fcb39b4e5fda4a, State: Initialized, Role: FOLLOWER
I20260925 00:47:08.029059 26091 consensus_queue.cc:260] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 } } peers { permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 } } peers { permanent_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46399 } }
I20260925 00:47:08.029147 26090 consensus_queue.cc:260] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 } } peers { permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 } } peers { permanent_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46399 } }
I20260925 00:47:08.029136 26092 consensus_queue.cc:260] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 } } peers { permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 } } peers { permanent_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46399 } }
I20260925 00:47:08.029718 25805 heartbeater.cc:503] Master 127.6.72.126:46205 was elected leader, sending a full tablet report...
I20260925 00:47:08.030097 26091 ts_tablet_manager.cc:1428] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 00:47:08.030104 26092 ts_tablet_manager.cc:1428] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 00:47:08.030104 26090 ts_tablet_manager.cc:1428] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 00:47:08.030441 25939 heartbeater.cc:503] Master 127.6.72.126:46205 was elected leader, sending a full tablet report...
I20260925 00:47:08.031463 26073 heartbeater.cc:503] Master 127.6.72.126:46205 was elected leader, sending a full tablet report...
I20260925 00:47:08.057428 26097 raft_consensus.cc:482] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:47:08.057631 26097 raft_consensus.cc:504] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 } } peers { permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 } } peers { permanent_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46399 } }
I20260925 00:47:08.058771 26097 leader_election.cc:288] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7865e662099d43be9efe6a9ec9d8b98f (127.6.72.65:41833), 571116e9c7384dc89a22b00acde5b9b0 (127.6.72.67:36203)
I20260925 00:47:08.063854 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.059651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:38731 (local address 127.6.72.67:36203)
0925 00:47:08.059805 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:08.059812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:08.060052 (+ 240us) server_negotiation.cc:407] Connection header received
0925 00:47:08.060145 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:08.060149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:08.060204 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:08.060348 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:08.060626 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.061284 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.062692 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.063300 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.063389 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.063444 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:08.063671 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 00:47:08.063740 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 00:47:08.063854 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.059261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:40927 (local address 127.6.72.65:41833)
0925 00:47:08.059401 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:08.059405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:08.059420 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:47:08.059586 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:08.059589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:08.059640 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:08.059724 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:08.060150 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.061001 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.062558 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.063213 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.063341 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.063415 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:08.063643 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 00:47:08.063722 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 00:47:08.063881 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.059173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:41833 (local address 127.6.72.66:40927)
0925 00:47:08.059319 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:08.059338 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:08.059448 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:08.059864 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:08.059867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:08.059887 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:08.060011 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.060021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.061132 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.061134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.062400 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.062413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.063338 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.063341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.063478 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.063537 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 00:47:08.063606 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 00:47:08.063722 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":31}
I20260925 00:47:08.063890 26099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.059511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:36203 (local address 127.6.72.66:38731)
0925 00:47:08.059946 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:08.059961 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:08.060062 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:08.060357 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:08.060364 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:08.060378 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:08.060468 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.060473 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.061388 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.061392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.062537 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.062544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.063399 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.063402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.063483 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.063526 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 00:47:08.063606 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 00:47:08.063732 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":46,"thread_start_us":165,"threads_started":1}
I20260925 00:47:08.064725 26027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36ab0d20c49447c49fe0b9f26f3773bc" candidate_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571116e9c7384dc89a22b00acde5b9b0" is_pre_election: true
I20260925 00:47:08.064725 25759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36ab0d20c49447c49fe0b9f26f3773bc" candidate_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7865e662099d43be9efe6a9ec9d8b98f" is_pre_election: true
I20260925 00:47:08.065053 25759 raft_consensus.cc:2454] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55cb87a0ef374a73a9fcb39b4e5fda4a in term 0.
I20260925 00:47:08.065053 26027 raft_consensus.cc:2454] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55cb87a0ef374a73a9fcb39b4e5fda4a in term 0.
I20260925 00:47:08.065557 25827 leader_election.cc:302] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cb87a0ef374a73a9fcb39b4e5fda4a, 7865e662099d43be9efe6a9ec9d8b98f; no voters:
I20260925 00:47:08.065836 26097 raft_consensus.cc:2781] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:47:08.065919 26097 raft_consensus.cc:482] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:47:08.065963 26097 raft_consensus.cc:3037] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:08.067023 26097 raft_consensus.cc:504] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 } } peers { permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 } } peers { permanent_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46399 } }
I20260925 00:47:08.067417 26097 leader_election.cc:288] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [CANDIDATE]: Term 1 election: Requested vote from peers 7865e662099d43be9efe6a9ec9d8b98f (127.6.72.65:41833), 571116e9c7384dc89a22b00acde5b9b0 (127.6.72.67:36203)
I20260925 00:47:08.067741 25759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36ab0d20c49447c49fe0b9f26f3773bc" candidate_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7865e662099d43be9efe6a9ec9d8b98f"
I20260925 00:47:08.067806 26027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36ab0d20c49447c49fe0b9f26f3773bc" candidate_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571116e9c7384dc89a22b00acde5b9b0"
I20260925 00:47:08.067883 25759 raft_consensus.cc:3037] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:08.067922 26027 raft_consensus.cc:3037] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:08.069248 26027 raft_consensus.cc:2454] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cb87a0ef374a73a9fcb39b4e5fda4a in term 1.
I20260925 00:47:08.069248 25759 raft_consensus.cc:2454] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55cb87a0ef374a73a9fcb39b4e5fda4a in term 1.
I20260925 00:47:08.069540 25827 leader_election.cc:302] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55cb87a0ef374a73a9fcb39b4e5fda4a, 7865e662099d43be9efe6a9ec9d8b98f; no voters:
I20260925 00:47:08.069727 26097 raft_consensus.cc:2781] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:08.069986 26097 raft_consensus.cc:686] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 1 LEADER]: Becoming Leader. State: Replica: 55cb87a0ef374a73a9fcb39b4e5fda4a, State: Running, Role: LEADER
I20260925 00:47:08.070583 26097 consensus_queue.cc:237] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 } } peers { permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 } } peers { permanent_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46399 } }
I20260925 00:47:08.074008 25613 catalog_manager.cc:5495] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 55cb87a0ef374a73a9fcb39b4e5fda4a (127.6.72.66). New cstate: current_term: 1 leader_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55cb87a0ef374a73a9fcb39b4e5fda4a" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 46399 } health_report { overall_health: HEALTHY } } }
I20260925 00:47:08.092550 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.088316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44184 (local address 127.6.72.66:46399)
0925 00:47:08.088481 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:08.088486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:08.088503 (+ 17us) server_negotiation.cc:407] Connection header received
0925 00:47:08.088677 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:08.088680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:08.088750 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:08.088887 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:08.089231 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.089811 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.090787 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.091039 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.091178 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.091524 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:08.091991 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:08.092141 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:08.092209 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 00:47:08.092309 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 00:47:08.099575 25759 raft_consensus.cc:1261] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f [term 1 FOLLOWER]: Refusing update from remote peer 55cb87a0ef374a73a9fcb39b4e5fda4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:08.099575 26027 raft_consensus.cc:1261] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0 [term 1 FOLLOWER]: Refusing update from remote peer 55cb87a0ef374a73a9fcb39b4e5fda4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:08.100404 26100 consensus_queue.cc:1035] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7865e662099d43be9efe6a9ec9d8b98f" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 41833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:08.100665 26097 consensus_queue.cc:1035] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "571116e9c7384dc89a22b00acde5b9b0" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 36203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:08.110833 26104 mvcc.cc:205] Tried to move back new op lower bound from 7333057446287777792 to 7333057446179700736. Current Snapshot: MvccSnapshot[applied={T|T < 7333057446287777792}]
I20260925 00:47:08.116802 25614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:08.117003 25614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:08.119670 25614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb98a44a50344917bbba8dff75c129de: Sending DeleteTablet for 3 replicas of tablet 36ab0d20c49447c49fe0b9f26f3773bc
I20260925 00:47:08.120515 25739 tablet_service.cc:1514] Processing DeleteTablet for tablet 36ab0d20c49447c49fe0b9f26f3773bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:08 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:57112
I20260925 00:47:08.120784 26007 tablet_service.cc:1514] Processing DeleteTablet for tablet 36ab0d20c49447c49fe0b9f26f3773bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:08 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:55866
I20260925 00:47:08.120832 25873 tablet_service.cc:1514] Processing DeleteTablet for tablet 36ab0d20c49447c49fe0b9f26f3773bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:08 UTC) from {username='kudu', principal='kudu/127.6.72.126@KRBTEST.COM'} at 127.0.0.1:44174
I20260925 00:47:08.121091 26112 tablet_replica.cc:330] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f: stopping tablet replica
I20260925 00:47:08.121440 26112 raft_consensus.cc:2229] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:08.121546 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 25676
I20260925 00:47:08.121829 26112 raft_consensus.cc:2258] T 36ab0d20c49447c49fe0b9f26f3773bc P 7865e662099d43be9efe6a9ec9d8b98f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:08.122268 26113 tablet_replica.cc:330] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: stopping tablet replica
I20260925 00:47:08.122591 26113 raft_consensus.cc:2229] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:08.122902 26114 tablet_replica.cc:330] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: stopping tablet replica
I20260925 00:47:08.123210 26114 raft_consensus.cc:2229] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:47:08.123586 26114 raft_consensus.cc:2258] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:08.124037 26113 raft_consensus.cc:2258] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:08.125782 26114 ts_tablet_manager.cc:1905] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:08.129390 26113 ts_tablet_manager.cc:1905] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:08.129556 26114 ts_tablet_manager.cc:1918] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:08.129671 26114 log.cc:1196] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/36ab0d20c49447c49fe0b9f26f3773bc
I20260925 00:47:08.130020 26114 ts_tablet_manager.cc:1939] T 36ab0d20c49447c49fe0b9f26f3773bc P 55cb87a0ef374a73a9fcb39b4e5fda4a: Deleting consensus metadata
W20260925 00:47:08.131695 25599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:41833 (error 108)
I20260925 00:47:08.132206 25599 catalog_manager.cc:4841] TS 55cb87a0ef374a73a9fcb39b4e5fda4a (127.6.72.66:46399): tablet 36ab0d20c49447c49fe0b9f26f3773bc (table test-table [id=aa1ad0ff32794bd69e8677e5a1622065]) successfully deleted
I20260925 00:47:08.132599 26113 ts_tablet_manager.cc:1918] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:08.132655 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 25810
I20260925 00:47:08.133479 26113 log.cc:1196] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/36ab0d20c49447c49fe0b9f26f3773bc
I20260925 00:47:08.133884 26113 ts_tablet_manager.cc:1939] T 36ab0d20c49447c49fe0b9f26f3773bc P 571116e9c7384dc89a22b00acde5b9b0: Deleting consensus metadata
I20260925 00:47:08.135126 25599 catalog_manager.cc:4841] TS 571116e9c7384dc89a22b00acde5b9b0 (127.6.72.67:36203): tablet 36ab0d20c49447c49fe0b9f26f3773bc (table test-table [id=aa1ad0ff32794bd69e8677e5a1622065]) successfully deleted
I20260925 00:47:08.135733 26084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.135459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:41833 (local address 127.0.0.1:57128)
0925 00:47:08.135586 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:08.135652 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:41833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260925 00:47:08.135880 25599 catalog_manager.cc:4584] TS 7865e662099d43be9efe6a9ec9d8b98f (127.6.72.65:41833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 36ab0d20c49447c49fe0b9f26f3773bc: Network error: Client connection negotiation failed: client connection to 127.6.72.65:41833: connect: Connection refused (error 111)
I20260925 00:47:08.138266 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 25944
I20260925 00:47:08.143707 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 25582
2026-09-25T00:47:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2310 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T00:47:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:08Z Disabled control of system clock
I20260925 00:47:08.175220 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:34601
--webserver_interface=127.6.72.126
--webserver_port=0
--builtin_ntp_servers=127.6.72.84:38315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:34601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 00:47:08.288110 26119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:08.288439 26119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:08.288478 26119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:08.292654 26119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:08.292730 26119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:08.292747 26119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:08.292764 26119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:08.292780 26119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:08.297672 26119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:34601
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:34601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:08.298801 26119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:08.300189 26119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:08.308844 26127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:08.308883 26124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:08.308965 26119 server_base.cc:1034] running on GCE node
W20260925 00:47:08.309180 26125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:08.309773 26119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:08.310947 26119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:08.312151 26119 hybrid_clock.cc:648] HybridClock initialized: now 1790297228312137 us; error 27 us; skew 500 ppm
I20260925 00:47:08.314067 26119 webserver.cc:460] Webserver started at http://127.6.72.126:45019/ using document root <none> and password file <none>
I20260925 00:47:08.314661 26119 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:08.314756 26119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:08.314992 26119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:08.317178 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "90329ad7d8c747ae847cfd5bbe937dd3"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "fe08e5ea2e13abafbd27c9e95bd9bc36"
server_key_iv: "3f6ee74f113a3452f2ddfa0286cb3770"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.317762 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "90329ad7d8c747ae847cfd5bbe937dd3"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "fe08e5ea2e13abafbd27c9e95bd9bc36"
server_key_iv: "3f6ee74f113a3452f2ddfa0286cb3770"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.321671 26119 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:47:08.324441 26133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:08.325932 26119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 00:47:08.326195 26119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "90329ad7d8c747ae847cfd5bbe937dd3"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "fe08e5ea2e13abafbd27c9e95bd9bc36"
server_key_iv: "3f6ee74f113a3452f2ddfa0286cb3770"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.326334 26119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:08.346522 26119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:08.347302 26119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:08.347513 26119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:08.355118 26119 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:34601
I20260925 00:47:08.355123 26185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:34601 every 8 connection(s)
I20260925 00:47:08.356134 26119 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:08.359442 26186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:08.361884 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 26119
I20260925 00:47:08.362021 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:08.362354 6433 external_mini_cluster.cc:1428] Setting key d422cfc004398185970de3c371f3961c
I20260925 00:47:08.366250 26186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3: Bootstrap starting.
I20260925 00:47:08.368664 26186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:08.369452 26186 log.cc:824] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:08.371922 26186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3: No bootstrap required, opened a new log
I20260925 00:47:08.372467 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.363921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.6.72.126:34601)
0925 00:47:08.364717 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:08.364734 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:08.364772 (+ 38us) server_negotiation.cc:407] Connection header received
0925 00:47:08.365441 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:08.365460 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:08.365898 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:08.366246 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:08.366684 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.367667 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.368644 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.369196 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.369374 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.370313 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:08.370318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:08.370332 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:08.370851 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:08.370923 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:08.370950 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:08.371052 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:08.371214 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 00:47:08.371496 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":151,"threads_started":1}
I20260925 00:47:08.376189 26186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90329ad7d8c747ae847cfd5bbe937dd3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34601 } }
I20260925 00:47:08.376480 26186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:08.376519 26186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90329ad7d8c747ae847cfd5bbe937dd3, State: Initialized, Role: FOLLOWER
I20260925 00:47:08.376895 26186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90329ad7d8c747ae847cfd5bbe937dd3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34601 } }
I20260925 00:47:08.377007 26186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:08.377092 26186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:08.377210 26186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:08.378161 26186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90329ad7d8c747ae847cfd5bbe937dd3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34601 } }
I20260925 00:47:08.378507 26186 leader_election.cc:302] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90329ad7d8c747ae847cfd5bbe937dd3; no voters:
I20260925 00:47:08.378823 26186 leader_election.cc:288] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:08.378962 26191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:08.379236 26191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 90329ad7d8c747ae847cfd5bbe937dd3, State: Running, Role: LEADER
I20260925 00:47:08.379542 26191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90329ad7d8c747ae847cfd5bbe937dd3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34601 } }
I20260925 00:47:08.380103 26186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:08.381800 26192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90329ad7d8c747ae847cfd5bbe937dd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90329ad7d8c747ae847cfd5bbe937dd3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34601 } } }
I20260925 00:47:08.381932 26192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:08.382182 26193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90329ad7d8c747ae847cfd5bbe937dd3. Latest consensus state: current_term: 1 leader_uuid: "90329ad7d8c747ae847cfd5bbe937dd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90329ad7d8c747ae847cfd5bbe937dd3" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 34601 } } }
I20260925 00:47:08.382264 26193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3 [sys.catalog]: This master's current role is: LEADER
W20260925 00:47:08.385170 26206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:08.385257 26206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:08.385317 26205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:08.386040 26205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:08.391683 26205 catalog_manager.cc:1347] Generated new cluster ID: e65cb47c4a72499f95f1827354be23f0
I20260925 00:47:08.391762 26205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:08.411975 26205 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:08.413477 26205 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:08.421777 26205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3: Generated new TSK 0
I20260925 00:47:08.422724 26205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:08.434685 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.65:0
--local_ip_for_outbound_sockets=127.6.72.65
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34601
--builtin_ntp_servers=127.6.72.84:38315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 00:47:08.552767 26210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:08.553113 26210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:08.553197 26210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:08.557303 26210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:08.557425 26210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:08.557535 26210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.65
I20260925 00:47:08.562502 26210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.72.65
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.65
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:08.563755 26210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:08.565284 26210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:08.575198 26216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:08.575198 26218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:08.575222 26215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:08.575292 26210 server_base.cc:1034] running on GCE node
I20260925 00:47:08.575843 26210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:08.576668 26210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:08.577883 26210 hybrid_clock.cc:648] HybridClock initialized: now 1790297228577861 us; error 33 us; skew 500 ppm
I20260925 00:47:08.579860 26210 webserver.cc:460] Webserver started at http://127.6.72.65:43441/ using document root <none> and password file <none>
I20260925 00:47:08.580579 26210 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:08.580680 26210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:08.580925 26210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:08.583192 26210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/data/instance:
uuid: "9e3f6eb18a8049c29eb2090d0b83025e"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "cf0e724f7c90ace9062a533ddcdcc820"
server_key_iv: "d29eb491010e9e207f33670da95e1666"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.583783 26210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance:
uuid: "9e3f6eb18a8049c29eb2090d0b83025e"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "cf0e724f7c90ace9062a533ddcdcc820"
server_key_iv: "d29eb491010e9e207f33670da95e1666"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.587802 26210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:47:08.590633 26224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:08.592010 26210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:47:08.592177 26210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/wal
uuid: "9e3f6eb18a8049c29eb2090d0b83025e"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "cf0e724f7c90ace9062a533ddcdcc820"
server_key_iv: "d29eb491010e9e207f33670da95e1666"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.592383 26210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:08.605820 26210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:08.606583 26210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:08.606813 26210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:08.607470 26210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:08.608551 26210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:08.608626 26210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:08.608700 26210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:08.608747 26210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:08.618865 26210 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.65:33515
I20260925 00:47:08.618896 26337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.65:33515 every 8 connection(s)
I20260925 00:47:08.619863 26210 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/data/info.pb
I20260925 00:47:08.621889 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 26210
I20260925 00:47:08.622046 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-0/wal/instance
I20260925 00:47:08.622371 6433 external_mini_cluster.cc:1428] Setting key e524586556ba86c32c007917f6f6e20a
I20260925 00:47:08.624501 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.66:0
--local_ip_for_outbound_sockets=127.6.72.66
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34601
--builtin_ntp_servers=127.6.72.84:38315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 00:47:08.630287 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.621848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:57093 (local address 127.6.72.126:34601)
0925 00:47:08.622015 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:08.622019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:08.623165 (+ 1146us) server_negotiation.cc:407] Connection header received
0925 00:47:08.624749 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:08.624754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:08.624814 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:08.624859 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:08.626123 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.626814 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.628091 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.628435 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.628551 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.629701 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:08.629704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:08.629707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:08.629721 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:08.629757 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:08.629765 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:08.629854 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:08.630105 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 00:47:08.630159 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 00:47:08.631871 26340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.622304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:34601 (local address 127.6.72.65:57093)
0925 00:47:08.623016 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:08.623069 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:08.624494 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:08.625194 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:08.625201 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:08.625299 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:08.625945 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.625969 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.626990 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.626992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.627961 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.627971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.628608 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.628613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.628777 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.629424 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:47:08.629426 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:08.629468 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:47:08.629535 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:08.629905 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:08.629921 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:08.629931 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 00:47:08.630111 (+ 180us) client_negotiation.cc:240] Negotiation successful
0925 00:47:08.630494 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":132,"threads_started":1}
I20260925 00:47:08.633332 26338 heartbeater.cc:347] Connected to a master server at 127.6.72.126:34601
I20260925 00:47:08.633642 26338 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:08.634393 26338 heartbeater.cc:511] Master 127.6.72.126:34601 requested a full tablet report, sending...
I20260925 00:47:08.636412 26150 ts_manager.cc:194] Registered new tserver with Master: 9e3f6eb18a8049c29eb2090d0b83025e (127.6.72.65:33515)
I20260925 00:47:08.638178 26150 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.65:57093
W20260925 00:47:08.739897 26341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:08.740167 26341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:08.740206 26341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:08.744297 26341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:08.744356 26341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:08.744411 26341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.66
I20260925 00:47:08.749262 26341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.72.66
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.66
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:08.750388 26341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:08.752159 26341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:08.761847 26346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:08.761850 26347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:08.762068 26341 server_base.cc:1034] running on GCE node
W20260925 00:47:08.762194 26349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:08.762605 26341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:08.763303 26341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:08.764497 26341 hybrid_clock.cc:648] HybridClock initialized: now 1790297228764481 us; error 25 us; skew 500 ppm
I20260925 00:47:08.770013 26341 webserver.cc:460] Webserver started at http://127.6.72.66:46445/ using document root <none> and password file <none>
I20260925 00:47:08.770658 26341 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:08.770758 26341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:08.771049 26341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:08.773310 26341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/data/instance:
uuid: "66d25fe556fd4c67b099db0e150943cf"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "41b517dd05d3c0c893a4f04af58b7973"
server_key_iv: "53125a93dffdcb721eb3be2866e1eb7b"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.773892 26341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance:
uuid: "66d25fe556fd4c67b099db0e150943cf"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "41b517dd05d3c0c893a4f04af58b7973"
server_key_iv: "53125a93dffdcb721eb3be2866e1eb7b"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.777704 26341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:47:08.780313 26355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:08.781718 26341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:47:08.781849 26341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/wal
uuid: "66d25fe556fd4c67b099db0e150943cf"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "41b517dd05d3c0c893a4f04af58b7973"
server_key_iv: "53125a93dffdcb721eb3be2866e1eb7b"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.781962 26341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:08.797699 26341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:08.798370 26341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:08.798563 26341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:08.799149 26341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:08.800076 26341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:08.800123 26341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:08.800199 26341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:08.800275 26341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:08.809089 26341 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.66:39815
I20260925 00:47:08.809131 26468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.66:39815 every 8 connection(s)
I20260925 00:47:08.809995 26341 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/data/info.pb
I20260925 00:47:08.811820 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 26341
I20260925 00:47:08.811959 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/wal/instance
I20260925 00:47:08.812361 6433 external_mini_cluster.cc:1428] Setting key 6b9f3df72ff9eae2b98eda60dfa15359
I20260925 00:47:08.814229 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.72.67:0
--local_ip_for_outbound_sockets=127.6.72.67
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34601
--builtin_ntp_servers=127.6.72.84:38315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 00:47:08.820425 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.811915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.66:60067 (local address 127.6.72.126:34601)
0925 00:47:08.812073 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:08.812078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:08.813362 (+ 1284us) server_negotiation.cc:407] Connection header received
0925 00:47:08.814807 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:08.814811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:08.814864 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:08.814946 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:08.815842 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.816488 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.817623 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.818050 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.818176 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.819786 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:08.819793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:08.819798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:08.819818 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:08.819847 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:08.819861 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:08.819951 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:08.820101 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 00:47:08.820179 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 00:47:08.821314 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.812478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:34601 (local address 127.6.72.66:60067)
0925 00:47:08.813193 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:08.813255 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:08.814595 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:08.815116 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:08.815124 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:08.815209 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:08.815692 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.815705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.816625 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.816637 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.817490 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.817499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.818256 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.818263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.818663 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:08.819469 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:47:08.819471 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:08.819510 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:47:08.819571 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:08.819954 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:08.819960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:08.819964 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 00:47:08.820057 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 00:47:08.820444 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":131,"threads_started":1}
I20260925 00:47:08.822700 26469 heartbeater.cc:347] Connected to a master server at 127.6.72.126:34601
I20260925 00:47:08.822939 26469 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:08.823448 26469 heartbeater.cc:511] Master 127.6.72.126:34601 requested a full tablet report, sending...
I20260925 00:47:08.824684 26150 ts_manager.cc:194] Registered new tserver with Master: 66d25fe556fd4c67b099db0e150943cf (127.6.72.66:39815)
I20260925 00:47:08.825542 26150 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.66:60067
W20260925 00:47:08.928543 26472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:08.928829 26472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:08.928870 26472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:08.933090 26472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:08.933154 26472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:08.933213 26472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.72.67
I20260925 00:47:08.937997 26472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:38315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.72.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.72.67
--webserver_port=0
--tserver_master_addrs=127.6.72.126:34601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.72.67
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:08.939049 26472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:08.940500 26472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:08.949882 26478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:08.949968 26480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:08.949899 26477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:08.950205 26472 server_base.cc:1034] running on GCE node
I20260925 00:47:08.950695 26472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:08.951304 26472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:08.952486 26472 hybrid_clock.cc:648] HybridClock initialized: now 1790297228952474 us; error 30 us; skew 500 ppm
I20260925 00:47:08.954236 26472 webserver.cc:460] Webserver started at http://127.6.72.67:40259/ using document root <none> and password file <none>
I20260925 00:47:08.954847 26472 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:08.954941 26472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:08.955159 26472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:08.957273 26472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/data/instance:
uuid: "1dda179daf094007b5af134c6a099b9b"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "871b09f69b2ee87c835eb480f5d4e2d6"
server_key_iv: "de8a2d07b61d3d8f8ed9d6b6cc94211a"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.957846 26472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance:
uuid: "1dda179daf094007b5af134c6a099b9b"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "871b09f69b2ee87c835eb480f5d4e2d6"
server_key_iv: "de8a2d07b61d3d8f8ed9d6b6cc94211a"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.961680 26472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:47:08.964092 26486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:08.965319 26472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:47:08.965456 26472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/wal
uuid: "1dda179daf094007b5af134c6a099b9b"
format_stamp: "Formatted at 2026-09-25 00:47:08 on dist-test-slave-w8nh"
server_key: "871b09f69b2ee87c835eb480f5d4e2d6"
server_key_iv: "de8a2d07b61d3d8f8ed9d6b6cc94211a"
server_key_version: "encryptionkey@0"
I20260925 00:47:08.965572 26472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:08.979485 26472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:08.980198 26472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:08.980444 26472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:08.981087 26472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 00:47:08.981966 26472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 00:47:08.982009 26472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:08.982048 26472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 00:47:08.982064 26472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:08.990759 26472 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.67:34373
I20260925 00:47:08.990796 26599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.67:34373 every 8 connection(s)
I20260925 00:47:08.991667 26472 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/data/info.pb
I20260925 00:47:09.000842 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 26472
I20260925 00:47:09.000993 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/wal/instance
I20260925 00:47:09.001354 6433 external_mini_cluster.cc:1428] Setting key ad3123dcb104c256a9749eaadffec8fc
I20260925 00:47:09.001709 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.993519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.67:46459 (local address 127.6.72.126:34601)
0925 00:47:08.993691 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:08.993694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:08.994807 (+ 1113us) server_negotiation.cc:407] Connection header received
0925 00:47:08.996436 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:08.996440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:08.996507 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:08.996601 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:08.997663 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.998228 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.999427 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.999691 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.999818 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.001160 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:09.001164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:09.001166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:09.001180 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:09.001201 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:09.001210 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:09.001280 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:09.001497 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 00:47:09.001572 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260925 00:47:09.002738 26602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:08.993938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:34601 (local address 127.6.72.67:46459)
0925 00:47:08.994643 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:08.994687 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:08.996151 (+ 1464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:08.996793 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:08.996800 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:08.996887 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:08.997493 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.997510 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.998370 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.998374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:08.999285 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:08.999295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:08.999856 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:08.999860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.000039 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.000753 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 00:47:09.000755 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:09.000814 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 00:47:09.000915 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:09.001310 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:09.001320 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:09.001327 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 00:47:09.001474 (+ 147us) client_negotiation.cc:240] Negotiation successful
0925 00:47:09.001811 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":107,"threads_started":1}
I20260925 00:47:09.003945 26600 heartbeater.cc:347] Connected to a master server at 127.6.72.126:34601
I20260925 00:47:09.004158 26600 heartbeater.cc:464] Registering TS with master...
I20260925 00:47:09.004659 26600 heartbeater.cc:511] Master 127.6.72.126:34601 requested a full tablet report, sending...
I20260925 00:47:09.005674 26149 ts_manager.cc:194] Registered new tserver with Master: 1dda179daf094007b5af134c6a099b9b (127.6.72.67:34373)
I20260925 00:47:09.006294 26149 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.72.67:46459
I20260925 00:47:09.015491 6433 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 00:47:09.021410 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.017483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.6.72.126:34601)
0925 00:47:09.017631 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:09.017637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:09.017849 (+ 212us) server_negotiation.cc:407] Connection header received
0925 00:47:09.018008 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:09.018011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:09.018063 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:09.018162 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:09.018438 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.018972 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.019829 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.020050 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.020162 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.020948 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:09.020952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:09.020955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:09.020975 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:09.020994 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:09.021002 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:09.021081 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:09.021195 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 00:47:09.021252 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:47:09.024224 26149 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 00:47:09.027197 26149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 00:47:09.041742 26612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.035561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:33515 (local address 127.0.0.1:56452)
0925 00:47:09.036140 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:09.036192 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:09.036369 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:09.037326 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:09.037332 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:09.037355 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:09.037489 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:09.037496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.038805 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.038809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.040259 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:09.040270 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.041191 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.041195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.041296 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.041331 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 00:47:09.041425 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 00:47:09.041547 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":65,"spinlock_wait_cycles":9472,"thread_start_us":86,"threads_started":1}
I20260925 00:47:09.041747 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.035745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.6.72.65:33515)
0925 00:47:09.036666 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:09.036679 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:09.036704 (+ 25us) server_negotiation.cc:407] Connection header received
0925 00:47:09.036782 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:09.036787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:09.037013 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:09.037181 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:09.037887 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.038676 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.040417 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.041068 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.041195 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.041396 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:09.041462 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 00:47:09.041555 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":129,"threads_started":1}
I20260925 00:47:09.041955 26611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.035740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:34373 (local address 127.0.0.1:51000)
0925 00:47:09.036141 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:09.036193 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:09.036368 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:09.037614 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:09.037620 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:09.037641 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:09.037744 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:09.037750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.039130 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.039134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.040358 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:09.040368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.041365 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.041369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.041460 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.041488 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 00:47:09.041539 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 00:47:09.041607 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":40,"spinlock_wait_cycles":4352,"thread_start_us":140,"threads_started":1}
I20260925 00:47:09.042483 26614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.036326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:39815 (local address 127.0.0.1:34748)
0925 00:47:09.037115 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:09.037130 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:09.037205 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:09.037818 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:09.037824 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:09.037842 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:09.037920 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:09.037925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.039211 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.039216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.040556 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:09.040566 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.041800 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.041803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.042179 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.042202 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 00:47:09.042291 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 00:47:09.042357 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":22,"thread_start_us":97,"threads_started":1}
I20260925 00:47:09.042706 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.036748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34748 (local address 127.6.72.66:39815)
0925 00:47:09.037088 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:09.037102 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:09.037293 (+ 191us) server_negotiation.cc:407] Connection header received
0925 00:47:09.037381 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:09.037385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:09.037519 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:09.037671 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:09.038038 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.039124 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.040690 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.041462 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.041563 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.041739 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:09.042336 (+ 597us) server_negotiation.cc:299] Negotiation successful
0925 00:47:09.042437 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":96,"threads_started":1}
I20260925 00:47:09.043022 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.036415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51000 (local address 127.6.72.67:34373)
0925 00:47:09.037183 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:09.037195 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:09.037221 (+ 26us) server_negotiation.cc:407] Connection header received
0925 00:47:09.037300 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:09.037305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:09.037471 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:09.037622 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:09.037853 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.039013 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.040459 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.041216 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.042188 (+ 972us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.042377 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:09.042427 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 00:47:09.042525 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":160,"threads_started":1}
I20260925 00:47:09.045068 26272 tablet_service.cc:1467] Processing CreateTablet for tablet aa31985e056641cfa0d3bce2cb20e578 (DEFAULT_TABLE table=test-table [id=84bb8c8038154797aa50dd93dd6925b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:09.045360 26403 tablet_service.cc:1467] Processing CreateTablet for tablet aa31985e056641cfa0d3bce2cb20e578 (DEFAULT_TABLE table=test-table [id=84bb8c8038154797aa50dd93dd6925b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:09.045585 26534 tablet_service.cc:1467] Processing CreateTablet for tablet aa31985e056641cfa0d3bce2cb20e578 (DEFAULT_TABLE table=test-table [id=84bb8c8038154797aa50dd93dd6925b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 00:47:09.046182 26272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa31985e056641cfa0d3bce2cb20e578. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:09.046391 26403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa31985e056641cfa0d3bce2cb20e578. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:09.046398 26534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa31985e056641cfa0d3bce2cb20e578. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:09.054123 26619 tablet_bootstrap.cc:492] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: Bootstrap starting.
I20260925 00:47:09.054131 26617 tablet_bootstrap.cc:492] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e: Bootstrap starting.
I20260925 00:47:09.056082 26617 tablet_bootstrap.cc:654] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:09.056082 26619 tablet_bootstrap.cc:654] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:09.056653 26618 tablet_bootstrap.cc:492] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: Bootstrap starting.
I20260925 00:47:09.056951 26617 log.cc:824] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:09.057003 26619 log.cc:824] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:09.058745 26618 tablet_bootstrap.cc:654] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:09.059134 26617 tablet_bootstrap.cc:492] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e: No bootstrap required, opened a new log
I20260925 00:47:09.059333 26617 ts_tablet_manager.cc:1397] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 00:47:09.059557 26618 log.cc:824] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:09.061681 26618 tablet_bootstrap.cc:492] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: No bootstrap required, opened a new log
I20260925 00:47:09.061887 26618 ts_tablet_manager.cc:1397] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260925 00:47:09.062877 26619 tablet_bootstrap.cc:492] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: No bootstrap required, opened a new log
I20260925 00:47:09.063077 26619 ts_tablet_manager.cc:1397] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260925 00:47:09.063758 26617 raft_consensus.cc:348] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 33515 } } peers { permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 } } peers { permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 } }
I20260925 00:47:09.064051 26617 raft_consensus.cc:374] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:09.064112 26617 raft_consensus.cc:729] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e3f6eb18a8049c29eb2090d0b83025e, State: Initialized, Role: FOLLOWER
I20260925 00:47:09.064616 26617 consensus_queue.cc:260] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 33515 } } peers { permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 } } peers { permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 } }
I20260925 00:47:09.065755 26617 ts_tablet_manager.cc:1428] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 00:47:09.066273 26338 heartbeater.cc:503] Master 127.6.72.126:34601 was elected leader, sending a full tablet report...
I20260925 00:47:09.066216 26618 raft_consensus.cc:348] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 33515 } } peers { permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 } } peers { permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 } }
I20260925 00:47:09.066504 26618 raft_consensus.cc:374] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:09.066606 26618 raft_consensus.cc:729] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d25fe556fd4c67b099db0e150943cf, State: Initialized, Role: FOLLOWER
I20260925 00:47:09.067057 26618 consensus_queue.cc:260] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 33515 } } peers { permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 } } peers { permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 } }
I20260925 00:47:09.067068 26619 raft_consensus.cc:348] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 33515 } } peers { permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 } } peers { permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 } }
I20260925 00:47:09.067307 26619 raft_consensus.cc:374] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:09.067363 26619 raft_consensus.cc:729] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dda179daf094007b5af134c6a099b9b, State: Initialized, Role: FOLLOWER
I20260925 00:47:09.067782 26619 consensus_queue.cc:260] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 33515 } } peers { permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 } } peers { permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 } }
I20260925 00:47:09.068044 26469 heartbeater.cc:503] Master 127.6.72.126:34601 was elected leader, sending a full tablet report...
I20260925 00:47:09.068400 26618 ts_tablet_manager.cc:1428] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:47:09.068743 26619 ts_tablet_manager.cc:1428] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 00:47:09.069062 26600 heartbeater.cc:503] Master 127.6.72.126:34601 was elected leader, sending a full tablet report...
W20260925 00:47:09.121130 26339 tablet.cc:2369] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:47:09.234297 26623 raft_consensus.cc:482] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 00:47:09.234563 26623 raft_consensus.cc:504] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 33515 } } peers { permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 } } peers { permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 } }
I20260925 00:47:09.235821 26623 leader_election.cc:288] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dda179daf094007b5af134c6a099b9b (127.6.72.67:34373), 66d25fe556fd4c67b099db0e150943cf (127.6.72.66:39815)
I20260925 00:47:09.241676 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.236196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:35387 (local address 127.6.72.67:34373)
0925 00:47:09.236361 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:09.236366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:09.236614 (+ 248us) server_negotiation.cc:407] Connection header received
0925 00:47:09.236748 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:09.236752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:09.236804 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:09.236890 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:09.237385 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.238271 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.240078 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.240869 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.240976 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.241034 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:09.241422 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 00:47:09.241517 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 00:47:09.241654 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.236271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.72.65:36719 (local address 127.6.72.66:39815)
0925 00:47:09.236481 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:09.236484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:09.236593 (+ 109us) server_negotiation.cc:407] Connection header received
0925 00:47:09.236749 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:09.236752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:09.236805 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:09.236890 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 00:47:09.237381 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.238288 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.240078 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.240834 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.240963 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.241034 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:09.241422 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 00:47:09.241520 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260925 00:47:09.241693 26626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.236099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.67:34373 (local address 127.6.72.65:35387)
0925 00:47:09.236517 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:09.236531 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:09.236633 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:09.236928 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:09.236931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:09.236959 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:09.237202 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:09.237212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.238427 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.238432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.239903 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:09.239917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.241003 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.241007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.241185 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.241278 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 00:47:09.241365 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 00:47:09.241496 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":283,"spinlock_wait_cycles":17664,"thread_start_us":104,"threads_started":1}
I20260925 00:47:09.241693 26627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.236099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.66:39815 (local address 127.6.72.65:36719)
0925 00:47:09.236468 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:09.236484 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:09.236633 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:09.236928 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:09.236931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:09.236957 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 00:47:09.237213 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:09.237219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.238427 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.238432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.239912 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:09.239917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.241003 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.241007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:09.241223 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.241283 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 00:47:09.241365 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 00:47:09.241494 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":315,"spinlock_wait_cycles":24064,"thread_start_us":111,"threads_started":1}
I20260925 00:47:09.242507 26423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa31985e056641cfa0d3bce2cb20e578" candidate_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66d25fe556fd4c67b099db0e150943cf" is_pre_election: true
I20260925 00:47:09.242578 26554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa31985e056641cfa0d3bce2cb20e578" candidate_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dda179daf094007b5af134c6a099b9b" is_pre_election: true
W20260925 00:47:09.242650 26601 tablet.cc:2369] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:47:09.242836 26423 raft_consensus.cc:2454] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e3f6eb18a8049c29eb2090d0b83025e in term 0.
I20260925 00:47:09.242885 26554 raft_consensus.cc:2454] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e3f6eb18a8049c29eb2090d0b83025e in term 0.
I20260925 00:47:09.243395 26227 leader_election.cc:302] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1dda179daf094007b5af134c6a099b9b, 66d25fe556fd4c67b099db0e150943cf, 9e3f6eb18a8049c29eb2090d0b83025e; no voters:
I20260925 00:47:09.243705 26623 raft_consensus.cc:2781] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 00:47:09.243798 26623 raft_consensus.cc:482] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 00:47:09.243842 26623 raft_consensus.cc:3037] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:09.244974 26623 raft_consensus.cc:504] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 33515 } } peers { permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 } } peers { permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 } }
I20260925 00:47:09.245329 26623 leader_election.cc:288] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [CANDIDATE]: Term 1 election: Requested vote from peers 1dda179daf094007b5af134c6a099b9b (127.6.72.67:34373), 66d25fe556fd4c67b099db0e150943cf (127.6.72.66:39815)
I20260925 00:47:09.245721 26423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa31985e056641cfa0d3bce2cb20e578" candidate_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66d25fe556fd4c67b099db0e150943cf"
I20260925 00:47:09.245774 26554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa31985e056641cfa0d3bce2cb20e578" candidate_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dda179daf094007b5af134c6a099b9b"
I20260925 00:47:09.245842 26423 raft_consensus.cc:3037] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:09.245937 26554 raft_consensus.cc:3037] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:09.246937 26423 raft_consensus.cc:2454] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e3f6eb18a8049c29eb2090d0b83025e in term 1.
I20260925 00:47:09.247104 26554 raft_consensus.cc:2454] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e3f6eb18a8049c29eb2090d0b83025e in term 1.
I20260925 00:47:09.247274 26227 leader_election.cc:302] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66d25fe556fd4c67b099db0e150943cf, 9e3f6eb18a8049c29eb2090d0b83025e; no voters:
I20260925 00:47:09.247519 26623 raft_consensus.cc:2781] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:09.247769 26623 raft_consensus.cc:686] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 1 LEADER]: Becoming Leader. State: Replica: 9e3f6eb18a8049c29eb2090d0b83025e, State: Running, Role: LEADER
I20260925 00:47:09.248157 26623 consensus_queue.cc:237] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 33515 } } peers { permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 } } peers { permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 } }
I20260925 00:47:09.251595 26150 catalog_manager.cc:5495] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e3f6eb18a8049c29eb2090d0b83025e (127.6.72.65). New cstate: current_term: 1 leader_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e3f6eb18a8049c29eb2090d0b83025e" member_type: VOTER last_known_addr { host: "127.6.72.65" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20260925 00:47:09.279712 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.275806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.6.72.65:33515)
0925 00:47:09.275976 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:09.275981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:09.275999 (+ 18us) server_negotiation.cc:407] Connection header received
0925 00:47:09.276137 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:09.276139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:09.276201 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:09.276360 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:09.276678 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.277208 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.278093 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:09.278319 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:09.278439 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:09.278786 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:09.279225 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:09.279364 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:09.279438 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 00:47:09.279499 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 00:47:09.286116 26554 raft_consensus.cc:1261] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b [term 1 FOLLOWER]: Refusing update from remote peer 9e3f6eb18a8049c29eb2090d0b83025e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:09.286116 26423 raft_consensus.cc:1261] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf [term 1 FOLLOWER]: Refusing update from remote peer 9e3f6eb18a8049c29eb2090d0b83025e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:09.286875 26628 consensus_queue.cc:1035] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [LEADER]: Connected to new peer: Peer: permanent_uuid: "66d25fe556fd4c67b099db0e150943cf" member_type: VOTER last_known_addr { host: "127.6.72.66" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:09.287092 26623 consensus_queue.cc:1035] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dda179daf094007b5af134c6a099b9b" member_type: VOTER last_known_addr { host: "127.6.72.67" port: 34373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:09.293977 26631 mvcc.cc:205] Tried to move back new op lower bound from 7333057451148038144 to 7333057451003428864. Current Snapshot: MvccSnapshot[applied={T|T < 7333057451148038144}]
I20260925 00:47:09.294440 26633 mvcc.cc:205] Tried to move back new op lower bound from 7333057451148038144 to 7333057451003428864. Current Snapshot: MvccSnapshot[applied={T|T < 7333057451148038144}]
I20260925 00:47:09.295825 26632 mvcc.cc:205] Tried to move back new op lower bound from 7333057451148038144 to 7333057451003428864. Current Snapshot: MvccSnapshot[applied={T|T < 7333057451148038144}]
I20260925 00:47:09.304112 26150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:09.304402 26150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 00:47:09.307184 26150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90329ad7d8c747ae847cfd5bbe937dd3: Sending DeleteTablet for 3 replicas of tablet aa31985e056641cfa0d3bce2cb20e578
I20260925 00:47:09.308079 26534 tablet_service.cc:1514] Processing DeleteTablet for tablet aa31985e056641cfa0d3bce2cb20e578 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:09 UTC) from {username='slave'} at 127.0.0.1:51000
I20260925 00:47:09.308177 26403 tablet_service.cc:1514] Processing DeleteTablet for tablet aa31985e056641cfa0d3bce2cb20e578 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:09 UTC) from {username='slave'} at 127.0.0.1:34748
I20260925 00:47:09.308693 26641 tablet_replica.cc:330] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: stopping tablet replica
I20260925 00:47:09.308712 26272 tablet_service.cc:1514] Processing DeleteTablet for tablet aa31985e056641cfa0d3bce2cb20e578 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 00:47:09 UTC) from {username='slave'} at 127.0.0.1:56452
I20260925 00:47:09.309070 26641 raft_consensus.cc:2229] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:09.309269 26642 tablet_replica.cc:330] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e: stopping tablet replica
I20260925 00:47:09.309449 26641 raft_consensus.cc:2258] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:09.309664 26642 raft_consensus.cc:2229] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 1 LEADER]: Raft consensus shutting down.
I20260925 00:47:09.309923 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 26210
I20260925 00:47:09.310181 26642 raft_consensus.cc:2258] T aa31985e056641cfa0d3bce2cb20e578 P 9e3f6eb18a8049c29eb2090d0b83025e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:09.312125 26640 tablet_replica.cc:330] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: stopping tablet replica
I20260925 00:47:09.312454 26640 raft_consensus.cc:2229] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 00:47:09.312767 26640 raft_consensus.cc:2258] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 00:47:09.313354 26641 ts_tablet_manager.cc:1905] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:09.314075 26640 ts_tablet_manager.cc:1905] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 00:47:09.316191 26641 ts_tablet_manager.cc:1918] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:09.316325 26641 log.cc:1196] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-2/wal/wals/aa31985e056641cfa0d3bce2cb20e578
I20260925 00:47:09.316648 26641 ts_tablet_manager.cc:1939] T aa31985e056641cfa0d3bce2cb20e578 P 1dda179daf094007b5af134c6a099b9b: Deleting consensus metadata
I20260925 00:47:09.316944 26640 ts_tablet_manager.cc:1918] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 00:47:09.317035 26640 log.cc:1196] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: Deleting WAL directory at /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790296971355272-6433-0/minicluster-data/ts-1/wal/wals/aa31985e056641cfa0d3bce2cb20e578
I20260925 00:47:09.317356 26640 ts_tablet_manager.cc:1939] T aa31985e056641cfa0d3bce2cb20e578 P 66d25fe556fd4c67b099db0e150943cf: Deleting consensus metadata
W20260925 00:47:09.317720 26136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.72.65:33515 (error 108)
I20260925 00:47:09.318069 26134 catalog_manager.cc:4841] TS 1dda179daf094007b5af134c6a099b9b (127.6.72.67:34373): tablet aa31985e056641cfa0d3bce2cb20e578 (table test-table [id=84bb8c8038154797aa50dd93dd6925b4]) successfully deleted
I20260925 00:47:09.318631 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 26341
I20260925 00:47:09.318679 26136 catalog_manager.cc:4841] TS 66d25fe556fd4c67b099db0e150943cf (127.6.72.66:39815): tablet aa31985e056641cfa0d3bce2cb20e578 (table test-table [id=84bb8c8038154797aa50dd93dd6925b4]) successfully deleted
I20260925 00:47:09.318912 26614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:09.318563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.65:33515 (local address 127.0.0.1:56462)
0925 00:47:09.318717 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:09.318808 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.65:33515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260925 00:47:09.319468 26136 catalog_manager.cc:4584] TS 9e3f6eb18a8049c29eb2090d0b83025e (127.6.72.65:33515): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aa31985e056641cfa0d3bce2cb20e578: Network error: Client connection negotiation failed: client connection to 127.6.72.65:33515: connect: Connection refused (error 111)
I20260925 00:47:09.324995 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 26472
I20260925 00:47:09.330937 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 26119
2026-09-25T00:47:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1185 ms)
[----------] 6 tests from EncryptionPolicyTest (10765 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:09 dist-test-slave-w8nh krb5kdc[26646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:09 dist-test-slave-w8nh krb5kdc[26646](info): set up 2 sockets
Sep 25 00:47:09 dist-test-slave-w8nh krb5kdc[26646](info): commencing operation
krb5kdc: starting...
W20260925 00:47:10.368462 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:10 dist-test-slave-w8nh krb5kdc[26646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:10.496116 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34565
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:10.608887 26659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:10.609238 26659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:10.609326 26659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:10.613507 26659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:10.613619 26659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:10.613660 26659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:10.613721 26659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:10.613757 26659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:10.619266 26659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34565
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:10.620455 26659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:10.621548 26659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:10.630167 26665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:10.630180 26667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:10.630180 26664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:10.630472 26659 server_base.cc:1034] running on GCE node
I20260925 00:47:10.631142 26659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:10.632381 26659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:10.633582 26659 hybrid_clock.cc:648] HybridClock initialized: now 1790297230633574 us; error 30 us; skew 500 ppm
Sep 25 00:47:10 dist-test-slave-w8nh krb5kdc[26646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297230, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:10.636533 26659 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:10.637426 26659 webserver.cc:460] Webserver started at http://127.0.0.1:46519/ using document root <none> and password file <none>
I20260925 00:47:10.637988 26659 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:10.638082 26659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:10.638321 26659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:10.640413 26659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "5f732d97e8bd4b62adf298adee7fdeab"
format_stamp: "Formatted at 2026-09-25 00:47:10 on dist-test-slave-w8nh"
server_key: "f9915d7741455a32ed87d24c6c4e066e"
server_key_iv: "0088b2fa4b2160c80c505eb3d5cc8a39"
server_key_version: "encryptionkey@0"
I20260925 00:47:10.640990 26659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "5f732d97e8bd4b62adf298adee7fdeab"
format_stamp: "Formatted at 2026-09-25 00:47:10 on dist-test-slave-w8nh"
server_key: "f9915d7741455a32ed87d24c6c4e066e"
server_key_iv: "0088b2fa4b2160c80c505eb3d5cc8a39"
server_key_version: "encryptionkey@0"
I20260925 00:47:10.644945 26659 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 00:47:10.647364 26674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:10.648695 26659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 00:47:10.648866 26659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "5f732d97e8bd4b62adf298adee7fdeab"
format_stamp: "Formatted at 2026-09-25 00:47:10 on dist-test-slave-w8nh"
server_key: "f9915d7741455a32ed87d24c6c4e066e"
server_key_iv: "0088b2fa4b2160c80c505eb3d5cc8a39"
server_key_version: "encryptionkey@0"
I20260925 00:47:10.648984 26659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:10.676671 26659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:10.679781 26659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:10.680002 26659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:10.687278 26659 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34565
I20260925 00:47:10.687299 26726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34565 every 8 connection(s)
I20260925 00:47:10.688282 26659 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:10.691118 26727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:10.692982 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 26659
I20260925 00:47:10.693164 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:10.693500 6433 external_mini_cluster.cc:1428] Setting key d3bb775d6b6f7018c7adf86646642c44
I20260925 00:47:10.697350 26727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab: Bootstrap starting.
I20260925 00:47:10.699546 26727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:10.700388 26727 log.cc:824] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:10.702759 26727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab: No bootstrap required, opened a new log
Sep 25 00:47:10 dist-test-slave-w8nh krb5kdc[26646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 00:47:10.706938 26727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f732d97e8bd4b62adf298adee7fdeab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } }
I20260925 00:47:10.707262 26727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:10.707326 26727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f732d97e8bd4b62adf298adee7fdeab, State: Initialized, Role: FOLLOWER
I20260925 00:47:10.707828 26727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f732d97e8bd4b62adf298adee7fdeab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } }
I20260925 00:47:10.708029 26727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:10.708109 26727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:10.708214 26727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:10.709463 26727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f732d97e8bd4b62adf298adee7fdeab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } }
I20260925 00:47:10.709832 26727 leader_election.cc:302] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f732d97e8bd4b62adf298adee7fdeab; no voters:
I20260925 00:47:10.710171 26727 leader_election.cc:288] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:10.710338 26732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:10.710601 26732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [term 1 LEADER]: Becoming Leader. State: Replica: 5f732d97e8bd4b62adf298adee7fdeab, State: Running, Role: LEADER
I20260925 00:47:10.710891 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:10.695009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57391 (local address 127.0.0.1:34565)
0925 00:47:10.695461 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:10.695477 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:10.695514 (+ 37us) server_negotiation.cc:407] Connection header received
0925 00:47:10.696515 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:10.696534 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:10.696911 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:10.697278 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:10.698167 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:10.699259 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:10.700308 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:10.700978 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:10.701176 (+ 198us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:10.704142 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:10.704148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:10.704166 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:10.704198 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:10.706973 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:10.707724 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:10.707729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:10.707735 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:10.707876 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:10.708491 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:10.708494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:10.708496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:10.709021 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:10.709244 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:10.709577 (+ 333us) server_negotiation.cc:299] Negotiation successful
0925 00:47:10.709814 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":121,"threads_started":1}
I20260925 00:47:10.710945 26732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f732d97e8bd4b62adf298adee7fdeab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } }
I20260925 00:47:10.711467 26727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:10.712795 26734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f732d97e8bd4b62adf298adee7fdeab. Latest consensus state: current_term: 1 leader_uuid: "5f732d97e8bd4b62adf298adee7fdeab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f732d97e8bd4b62adf298adee7fdeab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } } }
I20260925 00:47:10.712914 26734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:10.713032 26733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f732d97e8bd4b62adf298adee7fdeab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f732d97e8bd4b62adf298adee7fdeab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } } }
I20260925 00:47:10.713129 26733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:10.714807 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260925 00:47:10.716814 26747 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:10.716903 26747 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:10.717005 26741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:10.717869 26741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:10.723567 26741 catalog_manager.cc:1347] Generated new cluster ID: 3f5fa7eb13cd43bc9f61a6ec0821a489
I20260925 00:47:10.723634 26741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:10.727219 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:10.717042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53143 (local address 127.0.0.1:34565)
0925 00:47:10.717143 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:10.717147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:10.717486 (+ 339us) server_negotiation.cc:407] Connection header received
0925 00:47:10.717592 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:10.717595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:10.717649 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:10.717737 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:10.718686 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:10.719454 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:10.720888 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:10.721208 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:10.721320 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:10.723050 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:10.723057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:10.723061 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:10.723088 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:10.725406 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:10.725927 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:10.725932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:10.725939 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:10.726009 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:10.726337 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:10.726340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:10.726342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:10.726509 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:10.726678 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:10.726940 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 00:47:10.727039 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 00:47:10.749794 26741 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:10.751166 26741 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:10.761955 26741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f732d97e8bd4b62adf298adee7fdeab: Generated new TSK 0
I20260925 00:47:10.762909 26741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:10.777942 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 26659
W20260925 00:47:10.783528 26644 connection.cc:535] client connection to 127.0.0.1:34565 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34565): Connection reset by peer (error 104)
2026-09-25T00:47:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1445 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:10 dist-test-slave-w8nh krb5kdc[26758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:10 dist-test-slave-w8nh krb5kdc[26758](info): set up 2 sockets
Sep 25 00:47:10 dist-test-slave-w8nh krb5kdc[26758](info): commencing operation
krb5kdc: starting...
W20260925 00:47:11.900539 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:11 dist-test-slave-w8nh krb5kdc[26758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:12.026831 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43123
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:12.135766 26771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:12.136039 26771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:12.136082 26771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:12.140012 26771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:12.140074 26771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:12.140093 26771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:12.140110 26771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:12.140125 26771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:12.145426 26771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43123
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:12.146456 26771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:12.147419 26771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:12.156121 26777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:12.156121 26776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:12.156345 26771 server_base.cc:1034] running on GCE node
W20260925 00:47:12.156123 26779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:12.156936 26771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:12.158248 26771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:12.159451 26771 hybrid_clock.cc:648] HybridClock initialized: now 1790297232159435 us; error 32 us; skew 500 ppm
Sep 25 00:47:12 dist-test-slave-w8nh krb5kdc[26758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:12.162427 26771 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:12.163333 26771 webserver.cc:460] Webserver started at http://127.0.0.1:41151/ using document root <none> and password file <none>
I20260925 00:47:12.163920 26771 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:12.164013 26771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:12.164296 26771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:12.166383 26771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "99995a1c2dc84eddb712ba1822661177"
format_stamp: "Formatted at 2026-09-25 00:47:12 on dist-test-slave-w8nh"
server_key: "af74294a2e53f1c5fba2a08f38340517"
server_key_iv: "abe7a568484f84aa37ffccb4ef39b60e"
server_key_version: "encryptionkey@0"
I20260925 00:47:12.166976 26771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "99995a1c2dc84eddb712ba1822661177"
format_stamp: "Formatted at 2026-09-25 00:47:12 on dist-test-slave-w8nh"
server_key: "af74294a2e53f1c5fba2a08f38340517"
server_key_iv: "abe7a568484f84aa37ffccb4ef39b60e"
server_key_version: "encryptionkey@0"
I20260925 00:47:12.170871 26771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:47:12.173420 26786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:12.174783 26771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:47:12.174954 26771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "99995a1c2dc84eddb712ba1822661177"
format_stamp: "Formatted at 2026-09-25 00:47:12 on dist-test-slave-w8nh"
server_key: "af74294a2e53f1c5fba2a08f38340517"
server_key_iv: "abe7a568484f84aa37ffccb4ef39b60e"
server_key_version: "encryptionkey@0"
I20260925 00:47:12.175091 26771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:12.186105 26771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:12.189318 26771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:12.189530 26771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:12.197024 26771 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43123
I20260925 00:47:12.197096 26838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43123 every 8 connection(s)
I20260925 00:47:12.198120 26771 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:12.201087 26839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:12.203009 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 26771
I20260925 00:47:12.203158 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:12.203452 6433 external_mini_cluster.cc:1428] Setting key 855e03600479dbefd1888aa5121e2f3d
I20260925 00:47:12.207752 26839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177: Bootstrap starting.
I20260925 00:47:12.210088 26839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:12.210970 26839 log.cc:824] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:12.212723 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:12.204940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44927 (local address 127.0.0.1:43123)
0925 00:47:12.205438 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:12.205475 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:12.205525 (+ 50us) server_negotiation.cc:407] Connection header received
0925 00:47:12.206177 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:12.206195 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:12.206548 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:12.206970 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:12.207387 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:12.208481 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:12.209375 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:12.209834 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:12.210023 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:12.210936 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:12.210941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:12.210954 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:12.211406 (+ 452us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:12.211462 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:12.211486 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:12.211630 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:12.211809 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 00:47:12.211951 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":141,"threads_started":1}
I20260925 00:47:12.213083 26839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177: No bootstrap required, opened a new log
I20260925 00:47:12.215899 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:12.217576 26839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99995a1c2dc84eddb712ba1822661177" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } }
I20260925 00:47:12.217832 26839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:12.217939 26839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99995a1c2dc84eddb712ba1822661177, State: Initialized, Role: FOLLOWER
I20260925 00:47:12.218451 26839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99995a1c2dc84eddb712ba1822661177" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } }
I20260925 00:47:12.218627 26839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:12.218704 26839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:12.218816 26839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:12.220074 26839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99995a1c2dc84eddb712ba1822661177" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } }
I20260925 00:47:12.220492 26839 leader_election.cc:302] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99995a1c2dc84eddb712ba1822661177; no voters:
I20260925 00:47:12.220831 26839 leader_election.cc:288] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:12.220945 26849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:12.221196 26849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [term 1 LEADER]: Becoming Leader. State: Replica: 99995a1c2dc84eddb712ba1822661177, State: Running, Role: LEADER
I20260925 00:47:12.221580 26849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99995a1c2dc84eddb712ba1822661177" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } }
I20260925 00:47:12.222354 26839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:12.222680 26849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99995a1c2dc84eddb712ba1822661177. Latest consensus state: current_term: 1 leader_uuid: "99995a1c2dc84eddb712ba1822661177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99995a1c2dc84eddb712ba1822661177" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } } }
I20260925 00:47:12.222862 26849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:12.223136 26850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99995a1c2dc84eddb712ba1822661177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99995a1c2dc84eddb712ba1822661177" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43123 } } }
I20260925 00:47:12.223290 26850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:12.223332 26853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:12.223317 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:12.217950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47825 (local address 127.0.0.1:43123)
0925 00:47:12.218083 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:12.218088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:12.218300 (+ 212us) server_negotiation.cc:407] Connection header received
0925 00:47:12.218398 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:12.218401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:12.218448 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:12.218539 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:12.218896 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:12.219706 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:12.220679 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:12.221075 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:12.221195 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:12.222365 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:12.222373 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:12.222378 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:12.222393 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:12.222425 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:12.222442 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:12.222538 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:12.223050 (+ 512us) server_negotiation.cc:299] Negotiation successful
0925 00:47:12.223091 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 00:47:12.227028 26853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:12.232511 26853 catalog_manager.cc:1347] Generated new cluster ID: be2c4cff306b401593f321b1be191e33
I20260925 00:47:12.232595 26853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:12.247198 26853 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:12.248601 26853 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:12.260985 26853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99995a1c2dc84eddb712ba1822661177: Generated new TSK 0
I20260925 00:47:12.261844 26853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:12.269205 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 26771
W20260925 00:47:12.274748 26756 connection.cc:535] client connection to 127.0.0.1:43123 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43123): Connection reset by peer (error 104)
2026-09-25T00:47:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:12 dist-test-slave-w8nh krb5kdc[26870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:12 dist-test-slave-w8nh krb5kdc[26870](info): set up 2 sockets
Sep 25 00:47:12 dist-test-slave-w8nh krb5kdc[26870](info): commencing operation
krb5kdc: starting...
W20260925 00:47:13.312454 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:13 dist-test-slave-w8nh krb5kdc[26870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:13.440343 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:13.552595 26883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:13.552948 26883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:13.553035 26883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:13.557410 26883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:13.557545 26883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:13.557605 26883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:13.557663 26883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:13.557699 26883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:13.563530 26883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34775
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:13.564774 26883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:13.565912 26883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:13.574581 26889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:13.574573 26891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:13.574573 26888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:13.575023 26883 server_base.cc:1034] running on GCE node
I20260925 00:47:13.575565 26883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:13.576809 26883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:13.578008 26883 hybrid_clock.cc:648] HybridClock initialized: now 1790297233577995 us; error 26 us; skew 500 ppm
Sep 25 00:47:13 dist-test-slave-w8nh krb5kdc[26870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:13.581081 26883 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:13.582141 26883 webserver.cc:460] Webserver started at http://127.0.0.1:46755/ using document root <none> and password file <none>
I20260925 00:47:13.582738 26883 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:13.582831 26883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:13.583073 26883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:13.585659 26883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "d603b29c359949ddaea525eca688d743"
format_stamp: "Formatted at 2026-09-25 00:47:13 on dist-test-slave-w8nh"
server_key: "1d1bd116e4035e84cf2fdec5fe4416c8"
server_key_iv: "31bd32a841ddde10cfecd921436c2446"
server_key_version: "encryptionkey@0"
I20260925 00:47:13.586256 26883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "d603b29c359949ddaea525eca688d743"
format_stamp: "Formatted at 2026-09-25 00:47:13 on dist-test-slave-w8nh"
server_key: "1d1bd116e4035e84cf2fdec5fe4416c8"
server_key_iv: "31bd32a841ddde10cfecd921436c2446"
server_key_version: "encryptionkey@0"
I20260925 00:47:13.590284 26883 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:47:13.593004 26898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:13.594398 26883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:13.594568 26883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "d603b29c359949ddaea525eca688d743"
format_stamp: "Formatted at 2026-09-25 00:47:13 on dist-test-slave-w8nh"
server_key: "1d1bd116e4035e84cf2fdec5fe4416c8"
server_key_iv: "31bd32a841ddde10cfecd921436c2446"
server_key_version: "encryptionkey@0"
I20260925 00:47:13.594735 26883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:13.599561 26883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:13.603024 26883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:13.603242 26883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:13.610702 26883 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34775
I20260925 00:47:13.610718 26950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34775 every 8 connection(s)
I20260925 00:47:13.611675 26883 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:13.614538 26951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:13.617152 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 26883
I20260925 00:47:13.617347 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:13.617669 6433 external_mini_cluster.cc:1428] Setting key 3731fb3cce2974aee505f4efd46e3ce2
I20260925 00:47:13.621183 26951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743: Bootstrap starting.
I20260925 00:47:13.623306 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:13.619308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35047 (local address 127.0.0.1:34775)
0925 00:47:13.619819 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:13.619834 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:13.619871 (+ 37us) server_negotiation.cc:407] Connection header received
0925 00:47:13.620631 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:13.620652 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:13.620966 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:13.621327 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:13.622134 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:13.622138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:13.622150 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:13.622161 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:13.622214 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:13.622233 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:13.622381 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:13.622449 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 00:47:13.622611 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":165,"threads_started":1}
I20260925 00:47:13.623752 26951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:13.624609 26951 log.cc:824] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:13.626233 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:13.626762 26951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743: No bootstrap required, opened a new log
I20260925 00:47:13.630354 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:13.628459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38459 (local address 127.0.0.1:34775)
0925 00:47:13.628615 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:13.628620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:13.628789 (+ 169us) server_negotiation.cc:407] Connection header received
0925 00:47:13.628897 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:13.628900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:13.628936 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:13.629033 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:13.629887 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:13.629891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:13.629898 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:13.629913 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:13.629948 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:13.629961 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:13.630066 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:13.630127 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 00:47:13.630174 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 00:47:13.631453 26951 raft_consensus.cc:348] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d603b29c359949ddaea525eca688d743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34775 } }
I20260925 00:47:13.631695 26951 raft_consensus.cc:374] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:13.631783 26951 raft_consensus.cc:729] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d603b29c359949ddaea525eca688d743, State: Initialized, Role: FOLLOWER
I20260925 00:47:13.632222 26951 consensus_queue.cc:260] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d603b29c359949ddaea525eca688d743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34775 } }
I20260925 00:47:13.632417 26951 raft_consensus.cc:388] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:13.632463 26951 raft_consensus.cc:482] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:13.632563 26951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:13.633563 26951 raft_consensus.cc:504] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d603b29c359949ddaea525eca688d743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34775 } }
I20260925 00:47:13.633957 26951 leader_election.cc:302] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d603b29c359949ddaea525eca688d743; no voters:
I20260925 00:47:13.634363 26951 leader_election.cc:288] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:13.634472 26961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:13.634748 26961 raft_consensus.cc:686] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [term 1 LEADER]: Becoming Leader. State: Replica: d603b29c359949ddaea525eca688d743, State: Running, Role: LEADER
I20260925 00:47:13.635057 26961 consensus_queue.cc:237] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d603b29c359949ddaea525eca688d743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34775 } }
I20260925 00:47:13.635736 26951 sys_catalog.cc:564] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:13.636880 26962 sys_catalog.cc:455] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d603b29c359949ddaea525eca688d743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d603b29c359949ddaea525eca688d743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34775 } } }
I20260925 00:47:13.636873 26963 sys_catalog.cc:455] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d603b29c359949ddaea525eca688d743. Latest consensus state: current_term: 1 leader_uuid: "d603b29c359949ddaea525eca688d743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d603b29c359949ddaea525eca688d743" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34775 } } }
I20260925 00:47:13.637024 26962 sys_catalog.cc:458] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:13.637049 26963 sys_catalog.cc:458] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:13.637482 26970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:13.640619 26970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:13.646968 26970 catalog_manager.cc:1347] Generated new cluster ID: dcf1dbfca8094bfe96031132748127d8
I20260925 00:47:13.647059 26970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:13.661623 26970 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:13.661957 26970 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:13.670207 26970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d603b29c359949ddaea525eca688d743: Generated new TSK 0
I20260925 00:47:13.670998 26970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:13.686278 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 26883
2026-09-25T00:47:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:13 dist-test-slave-w8nh krb5kdc[26982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:13 dist-test-slave-w8nh krb5kdc[26982](info): set up 2 sockets
Sep 25 00:47:13 dist-test-slave-w8nh krb5kdc[26982](info): commencing operation
krb5kdc: starting...
W20260925 00:47:14.724628 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:14 dist-test-slave-w8nh krb5kdc[26982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:14.851506 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:14.968312 26995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:14.968600 26995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:14.968649 26995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:14.972926 26995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:14.972996 26995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:14.973016 26995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:14.973033 26995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:14.973048 26995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:14.978398 26995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:14.979485 26995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:14.980597 26995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:14.989218 27000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:14.989216 27001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:14.989547 26995 server_base.cc:1034] running on GCE node
W20260925 00:47:14.989233 27003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:14.990183 26995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:14.991302 26995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:14.992496 26995 hybrid_clock.cc:648] HybridClock initialized: now 1790297234992480 us; error 26 us; skew 500 ppm
Sep 25 00:47:14 dist-test-slave-w8nh krb5kdc[26982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297234, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:14.998608 26995 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:14.999599 26995 webserver.cc:460] Webserver started at http://127.0.0.1:44433/ using document root <none> and password file <none>
I20260925 00:47:15.000187 26995 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:15.000336 26995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:15.000588 26995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:15.002723 26995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "c3fc2a08c1914c1c840c23c1ec06976d"
format_stamp: "Formatted at 2026-09-25 00:47:14 on dist-test-slave-w8nh"
server_key: "1e0673b81c6bffb507497421d072d765"
server_key_iv: "4f80c863f47af314fbf8517d15c7b00c"
server_key_version: "encryptionkey@0"
I20260925 00:47:15.003299 26995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "c3fc2a08c1914c1c840c23c1ec06976d"
format_stamp: "Formatted at 2026-09-25 00:47:14 on dist-test-slave-w8nh"
server_key: "1e0673b81c6bffb507497421d072d765"
server_key_iv: "4f80c863f47af314fbf8517d15c7b00c"
server_key_version: "encryptionkey@0"
I20260925 00:47:15.007151 26995 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260925 00:47:15.010322 27010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:15.011579 26995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 00:47:15.011744 26995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "c3fc2a08c1914c1c840c23c1ec06976d"
format_stamp: "Formatted at 2026-09-25 00:47:14 on dist-test-slave-w8nh"
server_key: "1e0673b81c6bffb507497421d072d765"
server_key_iv: "4f80c863f47af314fbf8517d15c7b00c"
server_key_version: "encryptionkey@0"
I20260925 00:47:15.011875 26995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:15.025449 26995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:15.029639 26995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:15.029873 26995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:15.038646 26995 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46417
I20260925 00:47:15.038677 27062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46417 every 8 connection(s)
I20260925 00:47:15.039619 26995 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:15.042519 27063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:15.048152 27063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d: Bootstrap starting.
I20260925 00:47:15.048398 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 26995
I20260925 00:47:15.048547 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:15.048848 6433 external_mini_cluster.cc:1428] Setting key 342c59923641d59f2d635e0bfa58fd4f
I20260925 00:47:15.050876 27063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:15.051604 27063 log.cc:824] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:15.053468 27063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d: No bootstrap required, opened a new log
I20260925 00:47:15.057163 27063 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc2a08c1914c1c840c23c1ec06976d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46417 } }
I20260925 00:47:15.057466 27063 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:15.057524 27063 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3fc2a08c1914c1c840c23c1ec06976d, State: Initialized, Role: FOLLOWER
Sep 25 00:47:15 dist-test-slave-w8nh krb5kdc[26982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 00:47:15.057987 27063 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc2a08c1914c1c840c23c1ec06976d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46417 } }
I20260925 00:47:15.058116 27063 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:15.058213 27063 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:15.058326 27063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:15.059759 27063 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc2a08c1914c1c840c23c1ec06976d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46417 } }
I20260925 00:47:15.060151 27063 leader_election.cc:302] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3fc2a08c1914c1c840c23c1ec06976d; no voters:
I20260925 00:47:15.060542 27063 leader_election.cc:288] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:15.060638 27068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:15.060905 27068 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [term 1 LEADER]: Becoming Leader. State: Replica: c3fc2a08c1914c1c840c23c1ec06976d, State: Running, Role: LEADER
I20260925 00:47:15.061285 27068 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc2a08c1914c1c840c23c1ec06976d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46417 } }
I20260925 00:47:15.062080 27063 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:15.062492 27069 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3fc2a08c1914c1c840c23c1ec06976d. Latest consensus state: current_term: 1 leader_uuid: "c3fc2a08c1914c1c840c23c1ec06976d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc2a08c1914c1c840c23c1ec06976d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46417 } } }
I20260925 00:47:15.062688 27069 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:15.064208 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:15.050167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34795 (local address 127.0.0.1:46417)
0925 00:47:15.050600 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:15.050615 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:15.050649 (+ 34us) server_negotiation.cc:407] Connection header received
0925 00:47:15.051347 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:15.051366 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:15.051747 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:15.052073 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:15.052901 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:15.053918 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:15.054776 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:15.055253 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:15.055425 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:15.058314 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:15.058320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:15.058343 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:15.058390 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:15.060364 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:15.060932 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:15.060939 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:15.060946 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:15.061115 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:15.061500 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:15.061504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:15.061506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:15.062038 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:15.062278 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:15.062935 (+ 657us) server_negotiation.cc:299] Negotiation successful
0925 00:47:15.063240 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":146,"threads_started":1}
I20260925 00:47:15.065078 27070 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3fc2a08c1914c1c840c23c1ec06976d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fc2a08c1914c1c840c23c1ec06976d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46417 } } }
I20260925 00:47:15.065192 27070 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:15.065188 27072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:15.067415 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:15.068224 27072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:15.074028 27072 catalog_manager.cc:1347] Generated new cluster ID: 76320d82231647ddb00cddeaa8e36fca
I20260925 00:47:15.074162 27072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:15.079193 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:15.069735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54957 (local address 127.0.0.1:46417)
0925 00:47:15.069844 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:15.069848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:15.070172 (+ 324us) server_negotiation.cc:407] Connection header received
0925 00:47:15.070293 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:15.070296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:15.070346 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:15.070437 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:15.071256 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:15.072069 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:15.073354 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:15.073745 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:15.073823 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:15.075658 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:15.075661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:15.075664 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:15.075687 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:15.077346 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:15.077839 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:15.077842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:15.077844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:15.077922 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:15.078211 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:15.078214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:15.078216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:15.078439 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:15.078559 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:15.078897 (+ 338us) server_negotiation.cc:299] Negotiation successful
0925 00:47:15.079021 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":30}
I20260925 00:47:15.081019 27072 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:15.082226 27072 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:15.098892 27072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3fc2a08c1914c1c840c23c1ec06976d: Generated new TSK 0
I20260925 00:47:15.099820 27072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:15.109397 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 26995
2026-09-25T00:47:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:15 dist-test-slave-w8nh krb5kdc[27094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:15 dist-test-slave-w8nh krb5kdc[27094](info): set up 2 sockets
Sep 25 00:47:15 dist-test-slave-w8nh krb5kdc[27094](info): commencing operation
krb5kdc: starting...
W20260925 00:47:16.156629 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:16 dist-test-slave-w8nh krb5kdc[27094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:16.284534 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41217
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:16.397243 27107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:16.397528 27107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:16.397571 27107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:16.401686 27107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:16.401750 27107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:16.401769 27107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:16.401786 27107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:16.401801 27107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:16.407084 27107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:16.408111 27107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:16.409164 27107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:16.418292 27115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:16.418426 27107 server_base.cc:1034] running on GCE node
W20260925 00:47:16.418292 27113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:16.418318 27112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:16.419211 27107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:16.420410 27107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:16.421574 27107 hybrid_clock.cc:648] HybridClock initialized: now 1790297236421539 us; error 48 us; skew 500 ppm
Sep 25 00:47:16 dist-test-slave-w8nh krb5kdc[27094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:16.424232 27107 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:16.425241 27107 webserver.cc:460] Webserver started at http://127.0.0.1:46335/ using document root <none> and password file <none>
I20260925 00:47:16.425825 27107 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:16.425920 27107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:16.426151 27107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:16.428195 27107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "31a67fb79c5348aaa4dad5e33d2b1920"
format_stamp: "Formatted at 2026-09-25 00:47:16 on dist-test-slave-w8nh"
server_key: "f0f9934de699218710d9a6d3999195cf"
server_key_iv: "89c0205cde7001a475167e2c6fe6867f"
server_key_version: "encryptionkey@0"
I20260925 00:47:16.428828 27107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "31a67fb79c5348aaa4dad5e33d2b1920"
format_stamp: "Formatted at 2026-09-25 00:47:16 on dist-test-slave-w8nh"
server_key: "f0f9934de699218710d9a6d3999195cf"
server_key_iv: "89c0205cde7001a475167e2c6fe6867f"
server_key_version: "encryptionkey@0"
I20260925 00:47:16.432550 27107 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260925 00:47:16.434959 27122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:16.436200 27107 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:47:16.436406 27107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "31a67fb79c5348aaa4dad5e33d2b1920"
format_stamp: "Formatted at 2026-09-25 00:47:16 on dist-test-slave-w8nh"
server_key: "f0f9934de699218710d9a6d3999195cf"
server_key_iv: "89c0205cde7001a475167e2c6fe6867f"
server_key_version: "encryptionkey@0"
I20260925 00:47:16.436527 27107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:16.471238 27107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:16.474301 27107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:16.474520 27107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:16.482147 27107 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41217
I20260925 00:47:16.482174 27174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41217 every 8 connection(s)
I20260925 00:47:16.483199 27107 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:16.486073 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:16.491725 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920: Bootstrap starting.
I20260925 00:47:16.491741 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 27107
I20260925 00:47:16.491928 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:16.492205 6433 external_mini_cluster.cc:1428] Setting key dad3b967ccb30bad3af38cf9b3bbbfe5
I20260925 00:47:16.494576 27175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:16.495469 27175 log.cc:824] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:16.497988 27175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920: No bootstrap required, opened a new log
I20260925 00:47:16.502398 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:16.493648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51625 (local address 127.0.0.1:41217)
0925 00:47:16.494117 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:16.494136 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:16.494180 (+ 44us) server_negotiation.cc:407] Connection header received
0925 00:47:16.495093 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:16.495116 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:16.495554 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:16.495933 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:16.496334 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:16.497708 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:16.498635 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:16.499298 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:16.499478 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:16.500273 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:16.500280 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:16.500297 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:16.500836 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:16.500915 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:16.500950 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:16.501072 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:16.501183 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 00:47:16.501430 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":133,"threads_started":1}
I20260925 00:47:16.502650 27175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a67fb79c5348aaa4dad5e33d2b1920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } }
I20260925 00:47:16.502916 27175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:16.502983 27175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a67fb79c5348aaa4dad5e33d2b1920, State: Initialized, Role: FOLLOWER
I20260925 00:47:16.503404 27175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a67fb79c5348aaa4dad5e33d2b1920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } }
I20260925 00:47:16.503629 27175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:16.503697 27175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:16.503789 27175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:16.505069 27175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a67fb79c5348aaa4dad5e33d2b1920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } }
I20260925 00:47:16.505481 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:16.505431 27175 leader_election.cc:302] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31a67fb79c5348aaa4dad5e33d2b1920; no voters:
I20260925 00:47:16.505824 27175 leader_election.cc:288] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:16.505918 27180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:16.506141 27180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [term 1 LEADER]: Becoming Leader. State: Replica: 31a67fb79c5348aaa4dad5e33d2b1920, State: Running, Role: LEADER
I20260925 00:47:16.506517 27180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a67fb79c5348aaa4dad5e33d2b1920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } }
I20260925 00:47:16.507140 27175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:16.508232 27181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31a67fb79c5348aaa4dad5e33d2b1920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a67fb79c5348aaa4dad5e33d2b1920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } } }
I20260925 00:47:16.508394 27181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:16.508819 27193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:16.509057 27182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31a67fb79c5348aaa4dad5e33d2b1920. Latest consensus state: current_term: 1 leader_uuid: "31a67fb79c5348aaa4dad5e33d2b1920" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a67fb79c5348aaa4dad5e33d2b1920" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41217 } } }
I20260925 00:47:16.509241 27182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:16.512974 27193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:16.514674 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:16.507722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42065 (local address 127.0.0.1:41217)
0925 00:47:16.507846 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:16.507853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:16.509622 (+ 1769us) server_negotiation.cc:407] Connection header received
0925 00:47:16.509821 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:16.509825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:16.509876 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:16.509930 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:16.510383 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:16.511174 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:16.512324 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:16.512725 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:16.512815 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:16.514174 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:16.514178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:16.514181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:16.514195 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:16.514224 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:16.514238 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:16.514284 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:16.514445 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 00:47:16.514510 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":38}
I20260925 00:47:16.518743 27193 catalog_manager.cc:1347] Generated new cluster ID: baea81a8fab44260b1d2548f7d35ca35
I20260925 00:47:16.518821 27193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:16.546823 27193 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:16.548135 27193 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:16.563346 27193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31a67fb79c5348aaa4dad5e33d2b1920: Generated new TSK 0
I20260925 00:47:16.564313 27193 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:16.576418 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 27107
2026-09-25T00:47:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:16 dist-test-slave-w8nh krb5kdc[27206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:16 dist-test-slave-w8nh krb5kdc[27206](info): set up 2 sockets
Sep 25 00:47:16 dist-test-slave-w8nh krb5kdc[27206](info): commencing operation
krb5kdc: starting...
W20260925 00:47:17.616403 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:17 dist-test-slave-w8nh krb5kdc[27206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:17.741261 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:17.853842 27219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:17.854142 27219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:17.854187 27219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:17.858636 27219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:17.858704 27219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:17.858726 27219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:17.858769 27219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:17.858817 27219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:17.864323 27219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:17.865464 27219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:17.866508 27219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 00:47:17.875142 27219 server_base.cc:1034] running on GCE node
W20260925 00:47:17.875105 27227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:17.875129 27224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:17.875105 27225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:17.875891 27219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:17.877137 27219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:17.878321 27219 hybrid_clock.cc:648] HybridClock initialized: now 1790297237878302 us; error 33 us; skew 500 ppm
Sep 25 00:47:17 dist-test-slave-w8nh krb5kdc[27206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297237, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:17.880987 27219 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:17.881819 27219 webserver.cc:460] Webserver started at http://127.0.0.1:41025/ using document root <none> and password file <none>
I20260925 00:47:17.882359 27219 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:17.882448 27219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:17.882682 27219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:17.884737 27219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "2eca49f868db454fae08201c2fc474f1"
format_stamp: "Formatted at 2026-09-25 00:47:17 on dist-test-slave-w8nh"
server_key: "ff9970e776b8ec6ab51d826f44e81a67"
server_key_iv: "6a45189f32c73bce93076238b0862175"
server_key_version: "encryptionkey@0"
I20260925 00:47:17.885293 27219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "2eca49f868db454fae08201c2fc474f1"
format_stamp: "Formatted at 2026-09-25 00:47:17 on dist-test-slave-w8nh"
server_key: "ff9970e776b8ec6ab51d826f44e81a67"
server_key_iv: "6a45189f32c73bce93076238b0862175"
server_key_version: "encryptionkey@0"
I20260925 00:47:17.889009 27219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:17.891356 27234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:17.892665 27219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:47:17.892829 27219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "2eca49f868db454fae08201c2fc474f1"
format_stamp: "Formatted at 2026-09-25 00:47:17 on dist-test-slave-w8nh"
server_key: "ff9970e776b8ec6ab51d826f44e81a67"
server_key_iv: "6a45189f32c73bce93076238b0862175"
server_key_version: "encryptionkey@0"
I20260925 00:47:17.892942 27219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:17.897450 27219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:17.900447 27219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:17.900632 27219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:17.907629 27219 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39041
I20260925 00:47:17.907653 27286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39041 every 8 connection(s)
I20260925 00:47:17.908612 27219 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:17.911398 27287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:17.916903 27287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1: Bootstrap starting.
I20260925 00:47:17.917867 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 27219
I20260925 00:47:17.918045 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:17.918330 6433 external_mini_cluster.cc:1428] Setting key d5b35acd5c92c6409f37a8456ec2304d
I20260925 00:47:17.919687 27287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:17.920509 27287 log.cc:824] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:17.922729 27287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1: No bootstrap required, opened a new log
I20260925 00:47:17.923408 27290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:17.919464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48879 (local address 127.0.0.1:39041)
0925 00:47:17.919933 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:17.919947 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:17.919981 (+ 34us) server_negotiation.cc:407] Connection header received
0925 00:47:17.920750 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:17.920769 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:17.921045 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:17.921345 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:17.922144 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:17.922148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:17.922161 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:17.922170 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:17.922222 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:17.922240 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:17.922328 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:17.922461 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 00:47:17.922632 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":164,"threads_started":1}
I20260925 00:47:17.926194 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:17.926858 27287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eca49f868db454fae08201c2fc474f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39041 } }
I20260925 00:47:17.927101 27287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:17.927175 27287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eca49f868db454fae08201c2fc474f1, State: Initialized, Role: FOLLOWER
I20260925 00:47:17.927650 27287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eca49f868db454fae08201c2fc474f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39041 } }
I20260925 00:47:17.927799 27287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:17.927882 27287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:17.927979 27287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:17.929289 27287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eca49f868db454fae08201c2fc474f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39041 } }
I20260925 00:47:17.929690 27287 leader_election.cc:302] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eca49f868db454fae08201c2fc474f1; no voters:
I20260925 00:47:17.929953 27290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:17.928320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52225 (local address 127.0.0.1:39041)
0925 00:47:17.928428 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:17.928433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:17.928704 (+ 271us) server_negotiation.cc:407] Connection header received
0925 00:47:17.928762 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:17.928764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:17.928796 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:17.928867 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:17.929581 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:17.929584 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:17.929598 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:17.929609 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:17.929636 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:17.929648 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:17.929728 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:17.929802 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 00:47:17.929838 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 00:47:17.930099 27297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:17.930024 27287 leader_election.cc:288] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:17.930609 27297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [term 1 LEADER]: Becoming Leader. State: Replica: 2eca49f868db454fae08201c2fc474f1, State: Running, Role: LEADER
I20260925 00:47:17.930929 27297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eca49f868db454fae08201c2fc474f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39041 } }
I20260925 00:47:17.931425 27287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:17.932523 27298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eca49f868db454fae08201c2fc474f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eca49f868db454fae08201c2fc474f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39041 } } }
I20260925 00:47:17.932646 27298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:17.932940 27298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eca49f868db454fae08201c2fc474f1. Latest consensus state: current_term: 1 leader_uuid: "2eca49f868db454fae08201c2fc474f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eca49f868db454fae08201c2fc474f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39041 } } }
I20260925 00:47:17.933076 27298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:17.933125 27306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:17.936441 27306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:17.942750 27306 catalog_manager.cc:1347] Generated new cluster ID: a3c77a544a7443d8b664a2c11e51d879
I20260925 00:47:17.942823 27306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:17.951027 27306 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:17.951370 27306 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:17.958760 27306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2eca49f868db454fae08201c2fc474f1: Generated new TSK 0
I20260925 00:47:17.959513 27306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:17.968143 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 27219
2026-09-25T00:47:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:18 dist-test-slave-w8nh krb5kdc[27318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:18 dist-test-slave-w8nh krb5kdc[27318](info): set up 2 sockets
Sep 25 00:47:18 dist-test-slave-w8nh krb5kdc[27318](info): commencing operation
krb5kdc: starting...
W20260925 00:47:19.000564 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:19 dist-test-slave-w8nh krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:19.125469 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33541
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:19.240484 27331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:19.240845 27331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:19.240937 27331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:19.245198 27331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:19.245313 27331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:19.245352 27331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:19.245405 27331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:19.245442 27331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:19.251009 27331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:19.252274 27331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:19.253332 27331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:19.262661 27337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:19.262663 27339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:19.262738 27336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:19.262784 27331 server_base.cc:1034] running on GCE node
I20260925 00:47:19.263573 27331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:19.264843 27331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:19.266026 27331 hybrid_clock.cc:648] HybridClock initialized: now 1790297239265992 us; error 49 us; skew 500 ppm
Sep 25 00:47:19 dist-test-slave-w8nh krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:19.269029 27331 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:19.269994 27331 webserver.cc:460] Webserver started at http://127.0.0.1:37503/ using document root <none> and password file <none>
I20260925 00:47:19.270591 27331 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:19.270654 27331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:19.270884 27331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:19.273124 27331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "415bc9a8f4ea49e5a9b5671244e14500"
format_stamp: "Formatted at 2026-09-25 00:47:19 on dist-test-slave-w8nh"
server_key: "44131f61d512aec5f11cd3dd936563d8"
server_key_iv: "2b3f27c607cc84201864f81d54a0706f"
server_key_version: "encryptionkey@0"
I20260925 00:47:19.273736 27331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "415bc9a8f4ea49e5a9b5671244e14500"
format_stamp: "Formatted at 2026-09-25 00:47:19 on dist-test-slave-w8nh"
server_key: "44131f61d512aec5f11cd3dd936563d8"
server_key_iv: "2b3f27c607cc84201864f81d54a0706f"
server_key_version: "encryptionkey@0"
I20260925 00:47:19.277832 27331 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:47:19.280746 27346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:19.282121 27331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:19.282297 27331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "415bc9a8f4ea49e5a9b5671244e14500"
format_stamp: "Formatted at 2026-09-25 00:47:19 on dist-test-slave-w8nh"
server_key: "44131f61d512aec5f11cd3dd936563d8"
server_key_iv: "2b3f27c607cc84201864f81d54a0706f"
server_key_version: "encryptionkey@0"
I20260925 00:47:19.282433 27331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:19.332980 27331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:19.336580 27331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:19.336849 27331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:19.344131 27331 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33541
I20260925 00:47:19.344163 27398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33541 every 8 connection(s)
I20260925 00:47:19.345131 27331 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:19.347913 27399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:19.353610 27399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500: Bootstrap starting.
I20260925 00:47:19.354199 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 27331
I20260925 00:47:19.354357 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:19.354691 6433 external_mini_cluster.cc:1428] Setting key 6e39354bff3884efdb36f9f7b94f49f2
I20260925 00:47:19.356611 27399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:19.357513 27399 log.cc:824] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:19.359787 27399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500: No bootstrap required, opened a new log
I20260925 00:47:19.363916 27399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415bc9a8f4ea49e5a9b5671244e14500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } }
I20260925 00:47:19.364172 27399 raft_consensus.cc:374] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:19.364269 27399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 415bc9a8f4ea49e5a9b5671244e14500, State: Initialized, Role: FOLLOWER
Sep 25 00:47:19 dist-test-slave-w8nh krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 00:47:19.364786 27399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415bc9a8f4ea49e5a9b5671244e14500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } }
I20260925 00:47:19.364945 27399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:19.365042 27399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:19.365134 27399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:19.366077 27399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415bc9a8f4ea49e5a9b5671244e14500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } }
I20260925 00:47:19.366483 27399 leader_election.cc:302] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 415bc9a8f4ea49e5a9b5671244e14500; no voters:
I20260925 00:47:19.366847 27399 leader_election.cc:288] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:19.366967 27404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:19.367208 27404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [term 1 LEADER]: Becoming Leader. State: Replica: 415bc9a8f4ea49e5a9b5671244e14500, State: Running, Role: LEADER
I20260925 00:47:19.367604 27404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415bc9a8f4ea49e5a9b5671244e14500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } }
I20260925 00:47:19.368178 27399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:19.368952 27404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 415bc9a8f4ea49e5a9b5671244e14500. Latest consensus state: current_term: 1 leader_uuid: "415bc9a8f4ea49e5a9b5671244e14500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415bc9a8f4ea49e5a9b5671244e14500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } } }
I20260925 00:47:19.369127 27404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:19.369637 27406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "415bc9a8f4ea49e5a9b5671244e14500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415bc9a8f4ea49e5a9b5671244e14500" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } } }
I20260925 00:47:19.369745 27406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:19.371858 27402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:19.356060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:54843 (local address 127.0.0.1:33541)
0925 00:47:19.356616 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:19.356637 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:19.356682 (+ 45us) server_negotiation.cc:407] Connection header received
0925 00:47:19.357585 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:19.357611 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:19.358062 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:19.358421 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:19.359115 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:19.360616 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:19.361506 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:19.362210 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:19.362375 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:19.365063 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:19.365068 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:19.365081 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:19.365136 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:19.367264 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:19.367948 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:19.367954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:19.367959 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:19.368136 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:19.368563 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:19.368567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:19.368569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:19.369096 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:19.370007 (+ 911us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:19.370641 (+ 634us) server_negotiation.cc:299] Negotiation successful
0925 00:47:19.370904 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":190,"threads_started":1}
W20260925 00:47:19.374650 27419 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:19.374742 27419 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:19.374846 27413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:19.375027 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:19.375777 27413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:19.381817 27413 catalog_manager.cc:1347] Generated new cluster ID: 04e7fd8caf214581ab7fbfd71a59ce09
I20260925 00:47:19.381950 27413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:19.387066 27402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:19.377382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:59293 (local address 127.0.0.1:33541)
0925 00:47:19.377484 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:19.377489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:19.377772 (+ 283us) server_negotiation.cc:407] Connection header received
0925 00:47:19.377888 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:19.377891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:19.377941 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:19.378027 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:19.378921 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:19.379637 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:19.380656 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:19.380894 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:19.381005 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:19.382783 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:19.382788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:19.382791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:19.382820 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:19.384711 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:19.385305 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:19.385309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:19.385311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:19.385465 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:19.385898 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:19.385902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:19.385904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:19.386116 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:19.386290 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:19.386788 (+ 498us) server_negotiation.cc:299] Negotiation successful
0925 00:47:19.386881 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 00:47:19.393124 27413 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:19.394289 27413 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:19.403899 27413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 415bc9a8f4ea49e5a9b5671244e14500: Generated new TSK 0
I20260925 00:47:19.404742 27413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:19.409722 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 27331
W20260925 00:47:19.415313 27316 connection.cc:535] client connection to 127.0.0.1:33541 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33541): Connection reset by peer (error 104)
2026-09-25T00:47:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:19 dist-test-slave-w8nh krb5kdc[27430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:19 dist-test-slave-w8nh krb5kdc[27430](info): set up 2 sockets
Sep 25 00:47:19 dist-test-slave-w8nh krb5kdc[27430](info): commencing operation
krb5kdc: starting...
W20260925 00:47:20.456524 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:20 dist-test-slave-w8nh krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:20.583945 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:20.697640 27443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:20.697911 27443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:20.697953 27443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:20.702050 27443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:20.702121 27443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:20.702142 27443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:20.702158 27443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:20.702173 27443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:20.707468 27443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:20.708597 27443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:20.709643 27443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:20.718072 27451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:20.718253 27443 server_base.cc:1034] running on GCE node
W20260925 00:47:20.718070 27448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:20.718482 27449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:20.719059 27443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:20.720140 27443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:20.721298 27443 hybrid_clock.cc:648] HybridClock initialized: now 1790297240721283 us; error 29 us; skew 500 ppm
Sep 25 00:47:20 dist-test-slave-w8nh krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297240, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:20.730091 27443 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:20.731096 27443 webserver.cc:460] Webserver started at http://127.0.0.1:46445/ using document root <none> and password file <none>
I20260925 00:47:20.731649 27443 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:20.731711 27443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:20.731886 27443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:20.734018 27443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "253f22caf35449c8ac8d19fad09e794a"
format_stamp: "Formatted at 2026-09-25 00:47:20 on dist-test-slave-w8nh"
server_key: "49dd45b04dc26ba84f7b69e7d4bdafa0"
server_key_iv: "1b765dbf613f493eb30b5c106aecee06"
server_key_version: "encryptionkey@0"
I20260925 00:47:20.734524 27443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "253f22caf35449c8ac8d19fad09e794a"
format_stamp: "Formatted at 2026-09-25 00:47:20 on dist-test-slave-w8nh"
server_key: "49dd45b04dc26ba84f7b69e7d4bdafa0"
server_key_iv: "1b765dbf613f493eb30b5c106aecee06"
server_key_version: "encryptionkey@0"
I20260925 00:47:20.738261 27443 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260925 00:47:20.740767 27458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:20.741990 27443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:47:20.742136 27443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "253f22caf35449c8ac8d19fad09e794a"
format_stamp: "Formatted at 2026-09-25 00:47:20 on dist-test-slave-w8nh"
server_key: "49dd45b04dc26ba84f7b69e7d4bdafa0"
server_key_iv: "1b765dbf613f493eb30b5c106aecee06"
server_key_version: "encryptionkey@0"
I20260925 00:47:20.742254 27443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:20.752206 27443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:20.755265 27443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:20.755481 27443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:20.762663 27443 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37647
I20260925 00:47:20.762694 27510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37647 every 8 connection(s)
I20260925 00:47:20.763609 27443 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:20.766413 27511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:20.771413 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 27443
I20260925 00:47:20.771576 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:20.771834 6433 external_mini_cluster.cc:1428] Setting key 63f76f9a67e84182655143cdfe97858a
I20260925 00:47:20.771876 27511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a: Bootstrap starting.
I20260925 00:47:20.774369 27511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:20.775185 27511 log.cc:824] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:20.777354 27511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a: No bootstrap required, opened a new log
I20260925 00:47:20.781111 27514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:20.773233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:34363 (local address 127.0.0.1:37647)
0925 00:47:20.773658 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:20.773673 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:20.773709 (+ 36us) server_negotiation.cc:407] Connection header received
0925 00:47:20.774339 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:20.774356 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:20.774754 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:20.775041 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:20.775423 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:20.776622 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:20.777778 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:20.778237 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:20.778353 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:20.779373 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:20.779378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:20.779390 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:20.779781 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:20.779845 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:20.779867 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:20.779964 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:20.780182 (+ 218us) server_negotiation.cc:299] Negotiation successful
0925 00:47:20.780375 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":131,"threads_started":1}
I20260925 00:47:20.781563 27511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253f22caf35449c8ac8d19fad09e794a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } }
I20260925 00:47:20.781774 27511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:20.781831 27511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 253f22caf35449c8ac8d19fad09e794a, State: Initialized, Role: FOLLOWER
I20260925 00:47:20.782306 27511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253f22caf35449c8ac8d19fad09e794a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } }
I20260925 00:47:20.782455 27511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:20.782541 27511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:20.782660 27511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:20.783746 27511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253f22caf35449c8ac8d19fad09e794a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } }
I20260925 00:47:20.784147 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:20.784122 27511 leader_election.cc:302] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 253f22caf35449c8ac8d19fad09e794a; no voters:
I20260925 00:47:20.784513 27511 leader_election.cc:288] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:20.784603 27516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:20.784798 27516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [term 1 LEADER]: Becoming Leader. State: Replica: 253f22caf35449c8ac8d19fad09e794a, State: Running, Role: LEADER
I20260925 00:47:20.785077 27516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253f22caf35449c8ac8d19fad09e794a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } }
I20260925 00:47:20.785759 27511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:20.786242 27516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 253f22caf35449c8ac8d19fad09e794a. Latest consensus state: current_term: 1 leader_uuid: "253f22caf35449c8ac8d19fad09e794a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253f22caf35449c8ac8d19fad09e794a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } } }
I20260925 00:47:20.786350 27516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:20.786649 27517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "253f22caf35449c8ac8d19fad09e794a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253f22caf35449c8ac8d19fad09e794a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } } }
I20260925 00:47:20.786760 27517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a [sys.catalog]: This master's current role is: LEADER
W20260925 00:47:20.790527 27536 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:20.790607 27536 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:20.790685 27525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:20.791563 27525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:20.792603 27514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:20.786956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:39841 (local address 127.0.0.1:37647)
0925 00:47:20.787071 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:20.787077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:20.788317 (+ 1240us) server_negotiation.cc:407] Connection header received
0925 00:47:20.788505 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:20.788508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:20.788556 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:20.788638 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:20.789040 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:20.789789 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:20.790914 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:20.791175 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:20.791287 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:20.792144 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:20.792147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:20.792152 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:20.792169 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:20.792191 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:20.792200 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:20.792323 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:20.792437 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 00:47:20.792468 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 00:47:20.797009 27525 catalog_manager.cc:1347] Generated new cluster ID: 0622dc8fd6894398a1496be99fc00b05
I20260925 00:47:20.797072 27525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:20.819294 27525 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:20.821264 27525 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:20.831387 27525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 253f22caf35449c8ac8d19fad09e794a: Generated new TSK 0
I20260925 00:47:20.832208 27525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:20.837910 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 27443
W20260925 00:47:20.842563 27428 connection.cc:535] client connection to 127.0.0.1:37647 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37647): Connection reset by peer (error 104)
2026-09-25T00:47:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:20 dist-test-slave-w8nh krb5kdc[27542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:20 dist-test-slave-w8nh krb5kdc[27542](info): set up 2 sockets
Sep 25 00:47:20 dist-test-slave-w8nh krb5kdc[27542](info): commencing operation
krb5kdc: starting...
W20260925 00:47:21.932751 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:22 dist-test-slave-w8nh krb5kdc[27542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:22.063422 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:22.177526 27555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:22.177881 27555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:22.177969 27555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:22.182274 27555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:22.182384 27555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:22.182422 27555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:22.182459 27555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:22.182494 27555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:22.188009 27555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:22.189256 27555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:22.190315 27555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:22.199206 27563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:22.199194 27560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:22.199190 27561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:22.199654 27555 server_base.cc:1034] running on GCE node
I20260925 00:47:22.200196 27555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:22.201416 27555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:22.202625 27555 hybrid_clock.cc:648] HybridClock initialized: now 1790297242202611 us; error 28 us; skew 500 ppm
Sep 25 00:47:22 dist-test-slave-w8nh krb5kdc[27542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297242, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:22.205677 27555 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:22.206617 27555 webserver.cc:460] Webserver started at http://127.0.0.1:45883/ using document root <none> and password file <none>
I20260925 00:47:22.207158 27555 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:22.207221 27555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:22.207386 27555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:22.209628 27555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "ec92a1c611d540f3a974b282b446b3a3"
format_stamp: "Formatted at 2026-09-25 00:47:22 on dist-test-slave-w8nh"
server_key: "0362df05d20802fcfb5e91e4d148922d"
server_key_iv: "157e508dc165ac5f22af96a54bb7a9e4"
server_key_version: "encryptionkey@0"
I20260925 00:47:22.210153 27555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "ec92a1c611d540f3a974b282b446b3a3"
format_stamp: "Formatted at 2026-09-25 00:47:22 on dist-test-slave-w8nh"
server_key: "0362df05d20802fcfb5e91e4d148922d"
server_key_iv: "157e508dc165ac5f22af96a54bb7a9e4"
server_key_version: "encryptionkey@0"
I20260925 00:47:22.213967 27555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:47:22.216550 27570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:22.217872 27555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 00:47:22.218024 27555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "ec92a1c611d540f3a974b282b446b3a3"
format_stamp: "Formatted at 2026-09-25 00:47:22 on dist-test-slave-w8nh"
server_key: "0362df05d20802fcfb5e91e4d148922d"
server_key_iv: "157e508dc165ac5f22af96a54bb7a9e4"
server_key_version: "encryptionkey@0"
I20260925 00:47:22.218151 27555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:22.223088 27555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:22.226533 27555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:22.226802 27555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:22.234364 27555 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39671
I20260925 00:47:22.234391 27622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39671 every 8 connection(s)
I20260925 00:47:22.235339 27555 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:22.238220 27623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:22.240522 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 27555
I20260925 00:47:22.240664 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:22.240957 6433 external_mini_cluster.cc:1428] Setting key 2948f52ff82228d6d174bbcefb62b807
I20260925 00:47:22.244323 27623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3: Bootstrap starting.
I20260925 00:47:22.246265 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:22.242380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:54045 (local address 127.0.0.1:39671)
0925 00:47:22.242895 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:22.242911 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:22.242948 (+ 37us) server_negotiation.cc:407] Connection header received
0925 00:47:22.243606 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:22.243630 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:22.243919 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:22.244286 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:22.245098 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:22.245102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:22.245114 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:22.245125 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:22.245178 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:22.245198 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:22.245346 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:22.245417 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 00:47:22.245579 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":144,"threads_started":1}
I20260925 00:47:22.246783 27623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:22.247503 27623 log.cc:824] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:22.249186 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:22.249614 27623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3: No bootstrap required, opened a new log
I20260925 00:47:22.253237 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:22.251280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:39945 (local address 127.0.0.1:39671)
0925 00:47:22.251412 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:22.251417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:22.251616 (+ 199us) server_negotiation.cc:407] Connection header received
0925 00:47:22.251701 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:22.251703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:22.251734 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:22.251827 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:22.252691 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:22.252694 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:22.252696 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:22.252706 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:22.252731 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:22.252741 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:22.252822 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:22.252998 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 00:47:22.253082 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 00:47:22.254043 27623 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec92a1c611d540f3a974b282b446b3a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } }
I20260925 00:47:22.254302 27623 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:22.254369 27623 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec92a1c611d540f3a974b282b446b3a3, State: Initialized, Role: FOLLOWER
I20260925 00:47:22.254793 27623 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec92a1c611d540f3a974b282b446b3a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } }
I20260925 00:47:22.254942 27623 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:22.255010 27623 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:22.255108 27623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:22.256045 27623 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec92a1c611d540f3a974b282b446b3a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } }
I20260925 00:47:22.256424 27623 leader_election.cc:302] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec92a1c611d540f3a974b282b446b3a3; no voters:
I20260925 00:47:22.256770 27623 leader_election.cc:288] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:22.256883 27633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:22.257128 27633 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [term 1 LEADER]: Becoming Leader. State: Replica: ec92a1c611d540f3a974b282b446b3a3, State: Running, Role: LEADER
I20260925 00:47:22.257534 27633 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec92a1c611d540f3a974b282b446b3a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } }
I20260925 00:47:22.258074 27623 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:22.259403 27635 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec92a1c611d540f3a974b282b446b3a3. Latest consensus state: current_term: 1 leader_uuid: "ec92a1c611d540f3a974b282b446b3a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec92a1c611d540f3a974b282b446b3a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } } }
I20260925 00:47:22.259439 27634 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec92a1c611d540f3a974b282b446b3a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec92a1c611d540f3a974b282b446b3a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } } }
I20260925 00:47:22.259522 27635 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:22.259533 27634 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:22.259886 27643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:22.263039 27643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:22.268842 27643 catalog_manager.cc:1347] Generated new cluster ID: e9857ca4372b417f8b15f53f8a52dece
I20260925 00:47:22.268915 27643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:22.278806 27643 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:22.279178 27643 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:22.285058 27643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec92a1c611d540f3a974b282b446b3a3: Generated new TSK 0
I20260925 00:47:22.285804 27643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:22.290627 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 27555
2026-09-25T00:47:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:22 dist-test-slave-w8nh krb5kdc[27654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:22 dist-test-slave-w8nh krb5kdc[27654](info): set up 2 sockets
Sep 25 00:47:22 dist-test-slave-w8nh krb5kdc[27654](info): commencing operation
krb5kdc: starting...
W20260925 00:47:23.440671 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:23 dist-test-slave-w8nh krb5kdc[27654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:23.567910 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45851
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:23.679725 27667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:23.680078 27667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:23.680161 27667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:23.684358 27667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:23.684468 27667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:23.684514 27667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:23.684556 27667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:23.684589 27667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:23.690222 27667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45851
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:23.691421 27667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:23.692565 27667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:23.701059 27672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:23.701078 27675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:23.701059 27673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:23.701521 27667 server_base.cc:1034] running on GCE node
I20260925 00:47:23.702093 27667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:23.703260 27667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:23.704447 27667 hybrid_clock.cc:648] HybridClock initialized: now 1790297243704397 us; error 76 us; skew 500 ppm
Sep 25 00:47:23 dist-test-slave-w8nh krb5kdc[27654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:23.707217 27667 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:23.708151 27667 webserver.cc:460] Webserver started at http://127.0.0.1:34055/ using document root <none> and password file <none>
I20260925 00:47:23.708760 27667 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:23.708863 27667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:23.709100 27667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:23.711249 27667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "7b00b253fab24582aaf47f6b07b11f6d"
format_stamp: "Formatted at 2026-09-25 00:47:23 on dist-test-slave-w8nh"
server_key: "5ee416eaab1d936e90ec5c4db28f8627"
server_key_iv: "85446c1688e87521f0e443d3814d7a8b"
server_key_version: "encryptionkey@0"
I20260925 00:47:23.711817 27667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "7b00b253fab24582aaf47f6b07b11f6d"
format_stamp: "Formatted at 2026-09-25 00:47:23 on dist-test-slave-w8nh"
server_key: "5ee416eaab1d936e90ec5c4db28f8627"
server_key_iv: "85446c1688e87521f0e443d3814d7a8b"
server_key_version: "encryptionkey@0"
I20260925 00:47:23.715691 27667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:47:23.718209 27682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:23.719389 27667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:47:23.719528 27667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "7b00b253fab24582aaf47f6b07b11f6d"
format_stamp: "Formatted at 2026-09-25 00:47:23 on dist-test-slave-w8nh"
server_key: "5ee416eaab1d936e90ec5c4db28f8627"
server_key_iv: "85446c1688e87521f0e443d3814d7a8b"
server_key_version: "encryptionkey@0"
I20260925 00:47:23.719647 27667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:23.746716 27667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:23.749735 27667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:23.749958 27667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:23.757241 27667 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45851
I20260925 00:47:23.757267 27734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45851 every 8 connection(s)
I20260925 00:47:23.758268 27667 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:23.761152 27735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:23.765005 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 27667
I20260925 00:47:23.765132 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:23.765443 6433 external_mini_cluster.cc:1428] Setting key 74ce3cc08137b944bac6766798a5ac0d
I20260925 00:47:23.767481 27735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d: Bootstrap starting.
I20260925 00:47:23.769809 27735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:23.770601 27735 log.cc:824] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:23.772616 27735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d: No bootstrap required, opened a new log
Sep 25 00:47:23 dist-test-slave-w8nh krb5kdc[27654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 00:47:23.777055 27735 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b00b253fab24582aaf47f6b07b11f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } }
I20260925 00:47:23.777354 27735 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:23.777426 27735 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b00b253fab24582aaf47f6b07b11f6d, State: Initialized, Role: FOLLOWER
I20260925 00:47:23.777945 27735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b00b253fab24582aaf47f6b07b11f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } }
I20260925 00:47:23.778149 27735 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:23.778237 27735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:23.778362 27735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:23.779546 27735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b00b253fab24582aaf47f6b07b11f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } }
I20260925 00:47:23.779883 27735 leader_election.cc:302] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b00b253fab24582aaf47f6b07b11f6d; no voters:
I20260925 00:47:23.780190 27735 leader_election.cc:288] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:23.780313 27740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:23.780539 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:23.766944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:47889 (local address 127.0.0.1:45851)
0925 00:47:23.767392 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:23.767407 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:23.767444 (+ 37us) server_negotiation.cc:407] Connection header received
0925 00:47:23.768161 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:23.768180 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:23.768596 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:23.768933 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:23.769768 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:23.770887 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:23.771733 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:23.772212 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:23.772414 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:23.775258 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:23.775263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:23.775277 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:23.775315 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:23.777132 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:23.777788 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:23.777793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:23.777800 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:23.777948 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:23.778368 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:23.778371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:23.778373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:23.778814 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:23.778978 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:23.779308 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 00:47:23.779536 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":143,"threads_started":1}
I20260925 00:47:23.780536 27740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [term 1 LEADER]: Becoming Leader. State: Replica: 7b00b253fab24582aaf47f6b07b11f6d, State: Running, Role: LEADER
I20260925 00:47:23.781147 27740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b00b253fab24582aaf47f6b07b11f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } }
I20260925 00:47:23.781556 27735 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:23.783185 27741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b00b253fab24582aaf47f6b07b11f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b00b253fab24582aaf47f6b07b11f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } } }
I20260925 00:47:23.783314 27741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:23.784305 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:23.784379 27742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b00b253fab24582aaf47f6b07b11f6d. Latest consensus state: current_term: 1 leader_uuid: "7b00b253fab24582aaf47f6b07b11f6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b00b253fab24582aaf47f6b07b11f6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } } }
I20260925 00:47:23.784453 27742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d [sys.catalog]: This master's current role is: LEADER
W20260925 00:47:23.786551 27755 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:23.786661 27755 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:23.786763 27754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:23.787631 27754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:23.793838 27754 catalog_manager.cc:1347] Generated new cluster ID: 79904c2a157a4c378d06d4e0be4ab4ca
I20260925 00:47:23.793977 27754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:23.795270 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:23.786660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:35941 (local address 127.0.0.1:45851)
0925 00:47:23.786773 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:23.786780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:23.787112 (+ 332us) server_negotiation.cc:407] Connection header received
0925 00:47:23.787267 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:23.787270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:23.787318 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:23.787404 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:23.788048 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:23.788650 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:23.789445 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:23.789785 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:23.789890 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:23.791671 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:23.791680 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:23.791682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:23.791703 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:23.793409 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:23.793985 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:23.793989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:23.793990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:23.794060 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:23.794384 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:23.794387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:23.794389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:23.794569 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:23.794732 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:23.794999 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 00:47:23.795090 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 00:47:23.805512 27754 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:23.806454 27754 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:23.811170 27754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b00b253fab24582aaf47f6b07b11f6d: Generated new TSK 0
I20260925 00:47:23.811884 27754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:23.817417 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 27667
W20260925 00:47:23.822978 27652 connection.cc:535] client connection to 127.0.0.1:45851 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45851): Connection reset by peer (error 104)
2026-09-25T00:47:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1526 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:23 dist-test-slave-w8nh krb5kdc[27766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:23 dist-test-slave-w8nh krb5kdc[27766](info): set up 2 sockets
Sep 25 00:47:23 dist-test-slave-w8nh krb5kdc[27766](info): commencing operation
krb5kdc: starting...
W20260925 00:47:24.872669 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:24 dist-test-slave-w8nh krb5kdc[27766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:24.999387 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35855
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:25.113787 27779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:25.114140 27779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:25.114212 27779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:25.118395 27779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:25.118525 27779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:25.118566 27779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:25.118625 27779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:25.118666 27779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:25.124148 27779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35855
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:25.125334 27779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:25.126458 27779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:25.136682 27784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:25.136700 27785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:25.136837 27779 server_base.cc:1034] running on GCE node
W20260925 00:47:25.137014 27787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:25.137585 27779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:25.138753 27779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:25.139956 27779 hybrid_clock.cc:648] HybridClock initialized: now 1790297245139943 us; error 27 us; skew 500 ppm
Sep 25 00:47:25 dist-test-slave-w8nh krb5kdc[27766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297245, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:25.142731 27779 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:25.143666 27779 webserver.cc:460] Webserver started at http://127.0.0.1:36175/ using document root <none> and password file <none>
I20260925 00:47:25.144227 27779 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:25.144377 27779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:25.144619 27779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:25.146690 27779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "3775e3473a254fcda6d9790b6421419f"
format_stamp: "Formatted at 2026-09-25 00:47:25 on dist-test-slave-w8nh"
server_key: "4aaac1454533df36ed7772d2866579e8"
server_key_iv: "69300f9a0576df43d97c9ca55b862bf3"
server_key_version: "encryptionkey@0"
I20260925 00:47:25.147264 27779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "3775e3473a254fcda6d9790b6421419f"
format_stamp: "Formatted at 2026-09-25 00:47:25 on dist-test-slave-w8nh"
server_key: "4aaac1454533df36ed7772d2866579e8"
server_key_iv: "69300f9a0576df43d97c9ca55b862bf3"
server_key_version: "encryptionkey@0"
I20260925 00:47:25.151047 27779 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260925 00:47:25.153676 27794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:25.154922 27779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:47:25.155059 27779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "3775e3473a254fcda6d9790b6421419f"
format_stamp: "Formatted at 2026-09-25 00:47:25 on dist-test-slave-w8nh"
server_key: "4aaac1454533df36ed7772d2866579e8"
server_key_iv: "69300f9a0576df43d97c9ca55b862bf3"
server_key_version: "encryptionkey@0"
I20260925 00:47:25.155172 27779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:25.169104 27779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:25.172331 27779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:25.172536 27779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:25.179725 27779 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35855
I20260925 00:47:25.179751 27846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35855 every 8 connection(s)
I20260925 00:47:25.180719 27779 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:25.183691 27847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:25.186301 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 27779
I20260925 00:47:25.186466 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:25.186774 6433 external_mini_cluster.cc:1428] Setting key 6080eb6f6f19f51cc75d58f8ac4f53c2
I20260925 00:47:25.190701 27847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f: Bootstrap starting.
I20260925 00:47:25.193167 27847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:25.193962 27847 log.cc:824] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:25.196061 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:25.188335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:48537 (local address 127.0.0.1:35855)
0925 00:47:25.188781 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:25.188799 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:25.188835 (+ 36us) server_negotiation.cc:407] Connection header received
0925 00:47:25.189584 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:25.189603 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:25.190063 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:25.190404 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:25.190829 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:25.191805 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:25.192758 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:25.193252 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:25.193448 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:25.194416 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:25.194422 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:25.194434 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:25.194818 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:25.194871 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:25.194893 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:25.195028 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:25.195200 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 00:47:25.195368 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":139,"threads_started":1}
I20260925 00:47:25.196141 27847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f: No bootstrap required, opened a new log
I20260925 00:47:25.199051 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:25.200456 27847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3775e3473a254fcda6d9790b6421419f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35855 } }
I20260925 00:47:25.200699 27847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:25.200778 27847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3775e3473a254fcda6d9790b6421419f, State: Initialized, Role: FOLLOWER
I20260925 00:47:25.201316 27847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3775e3473a254fcda6d9790b6421419f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35855 } }
I20260925 00:47:25.201509 27847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:25.201597 27847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:25.201725 27847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:25.202732 27847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3775e3473a254fcda6d9790b6421419f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35855 } }
I20260925 00:47:25.203089 27847 leader_election.cc:302] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3775e3473a254fcda6d9790b6421419f; no voters:
I20260925 00:47:25.203449 27847 leader_election.cc:288] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:25.203523 27857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:25.203770 27857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [term 1 LEADER]: Becoming Leader. State: Replica: 3775e3473a254fcda6d9790b6421419f, State: Running, Role: LEADER
I20260925 00:47:25.204190 27857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3775e3473a254fcda6d9790b6421419f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35855 } }
I20260925 00:47:25.204892 27847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:25.205579 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:25.201095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:55585 (local address 127.0.0.1:35855)
0925 00:47:25.201205 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:25.201210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:25.201387 (+ 177us) server_negotiation.cc:407] Connection header received
0925 00:47:25.201463 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:25.201466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:25.201520 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:25.201622 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:25.201920 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:25.202442 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:25.203352 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:25.203699 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:25.203822 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:25.204914 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:25.204920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:25.204924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:25.204938 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:25.204963 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:25.204973 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:25.205046 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:25.205228 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 00:47:25.205412 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 00:47:25.205982 27858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3775e3473a254fcda6d9790b6421419f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3775e3473a254fcda6d9790b6421419f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35855 } } }
I20260925 00:47:25.206096 27858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:25.206452 27859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3775e3473a254fcda6d9790b6421419f. Latest consensus state: current_term: 1 leader_uuid: "3775e3473a254fcda6d9790b6421419f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3775e3473a254fcda6d9790b6421419f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35855 } } }
I20260925 00:47:25.206712 27859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:25.207436 27866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:25.210779 27866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:25.217382 27866 catalog_manager.cc:1347] Generated new cluster ID: 77519b05140f49b2b249e27cb9d7ece6
I20260925 00:47:25.217542 27866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:25.234432 27866 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:25.235760 27866 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:25.241503 27866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3775e3473a254fcda6d9790b6421419f: Generated new TSK 0
I20260925 00:47:25.242369 27866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:25.255759 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 27779
W20260925 00:47:25.261375 27764 connection.cc:535] client connection to 127.0.0.1:35855 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35855): Connection reset by peer (error 104)
2026-09-25T00:47:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:25 dist-test-slave-w8nh krb5kdc[27878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:25 dist-test-slave-w8nh krb5kdc[27878](info): set up 2 sockets
Sep 25 00:47:25 dist-test-slave-w8nh krb5kdc[27878](info): commencing operation
krb5kdc: starting...
W20260925 00:47:26.340600 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:26 dist-test-slave-w8nh krb5kdc[27878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 00:47:26.470994 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:26.591109 27891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:26.591429 27891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:26.591477 27891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:26.595867 27891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:26.595973 27891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:26.595997 27891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:26.596019 27891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:26.596037 27891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:26.601732 27891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41039
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:26.602916 27891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:26.603987 27891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:26.613356 27896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:26.613385 27897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:26.613449 27891 server_base.cc:1034] running on GCE node
W20260925 00:47:26.613385 27899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:26.614295 27891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:26.615517 27891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:26.616709 27891 hybrid_clock.cc:648] HybridClock initialized: now 1790297246616698 us; error 28 us; skew 500 ppm
Sep 25 00:47:26 dist-test-slave-w8nh krb5kdc[27878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297246, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:26.623867 27891 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 00:47:26.624956 27891 webserver.cc:460] Webserver started at http://127.0.0.1:39669/ using document root <none> and password file <none>
I20260925 00:47:26.625582 27891 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:26.625680 27891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:26.625926 27891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:26.628099 27891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "205ebe4386f648e5967e82c48e602472"
format_stamp: "Formatted at 2026-09-25 00:47:26 on dist-test-slave-w8nh"
server_key: "a0f140f5a2c6a1776d8a2024fe55ccb6"
server_key_iv: "4cd0bce9be563554b03f52d3da4aafb8"
server_key_version: "encryptionkey@0"
I20260925 00:47:26.628751 27891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "205ebe4386f648e5967e82c48e602472"
format_stamp: "Formatted at 2026-09-25 00:47:26 on dist-test-slave-w8nh"
server_key: "a0f140f5a2c6a1776d8a2024fe55ccb6"
server_key_iv: "4cd0bce9be563554b03f52d3da4aafb8"
server_key_version: "encryptionkey@0"
I20260925 00:47:26.632860 27891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260925 00:47:26.635489 27906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:26.636907 27891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:26.637063 27891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "205ebe4386f648e5967e82c48e602472"
format_stamp: "Formatted at 2026-09-25 00:47:26 on dist-test-slave-w8nh"
server_key: "a0f140f5a2c6a1776d8a2024fe55ccb6"
server_key_iv: "4cd0bce9be563554b03f52d3da4aafb8"
server_key_version: "encryptionkey@0"
I20260925 00:47:26.637187 27891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:26.642350 27891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:26.645741 27891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:26.645951 27891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:26.653388 27891 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41039
I20260925 00:47:26.653412 27958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41039 every 8 connection(s)
I20260925 00:47:26.654430 27891 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:26.657560 27959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:26.658198 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 27891
I20260925 00:47:26.658358 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:26.658710 6433 external_mini_cluster.cc:1428] Setting key 8adb6adf88ec8b5d47a00a0ed47fe69c
I20260925 00:47:26.664187 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:26.660283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:37399 (local address 127.0.0.1:41039)
0925 00:47:26.660776 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:26.660792 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:26.660827 (+ 35us) server_negotiation.cc:407] Connection header received
0925 00:47:26.661483 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:26.661501 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:26.661805 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:26.662154 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:26.662994 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:26.662998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:26.663011 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:26.663021 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:26.663078 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:26.663099 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:26.663251 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:26.663324 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 00:47:26.663488 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":132,"threads_started":1}
I20260925 00:47:26.665453 27959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472: Bootstrap starting.
I20260925 00:47:26.667335 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:26.668129 27959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:26.669025 27959 log.cc:824] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:26.671180 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:26.669447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.77:51091 (local address 127.0.0.1:41039)
0925 00:47:26.669577 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:26.669581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:26.669753 (+ 172us) server_negotiation.cc:407] Connection header received
0925 00:47:26.669856 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:26.669858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:26.669889 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:26.669982 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:26.670724 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:26.670727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:26.670729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:26.670740 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:26.670769 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:26.670782 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:26.670856 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:26.670969 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 00:47:26.671006 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 00:47:26.671883 27959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472: No bootstrap required, opened a new log
I20260925 00:47:26.675568 27959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205ebe4386f648e5967e82c48e602472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41039 } }
I20260925 00:47:26.675812 27959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:26.675881 27959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 205ebe4386f648e5967e82c48e602472, State: Initialized, Role: FOLLOWER
I20260925 00:47:26.676378 27959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205ebe4386f648e5967e82c48e602472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41039 } }
I20260925 00:47:26.676534 27959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:26.676605 27959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:26.676693 27959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:26.677816 27959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205ebe4386f648e5967e82c48e602472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41039 } }
I20260925 00:47:26.678117 27959 leader_election.cc:302] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 205ebe4386f648e5967e82c48e602472; no voters:
I20260925 00:47:26.678408 27959 leader_election.cc:288] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:26.678589 27969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:26.678920 27969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [term 1 LEADER]: Becoming Leader. State: Replica: 205ebe4386f648e5967e82c48e602472, State: Running, Role: LEADER
I20260925 00:47:26.679245 27969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205ebe4386f648e5967e82c48e602472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41039 } }
I20260925 00:47:26.679677 27959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:26.680563 27969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 205ebe4386f648e5967e82c48e602472. Latest consensus state: current_term: 1 leader_uuid: "205ebe4386f648e5967e82c48e602472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205ebe4386f648e5967e82c48e602472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41039 } } }
I20260925 00:47:26.680696 27969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:26.681102 27970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "205ebe4386f648e5967e82c48e602472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205ebe4386f648e5967e82c48e602472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41039 } } }
I20260925 00:47:26.681202 27970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472 [sys.catalog]: This master's current role is: LEADER
W20260925 00:47:26.684844 27984 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:26.684932 27984 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:26.684993 27978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:26.685856 27978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:26.691300 27978 catalog_manager.cc:1347] Generated new cluster ID: ca1ac78d93f5412c870f1767cfe959ee
I20260925 00:47:26.691372 27978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:26.703907 27978 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:26.704646 27978 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:26.718009 27978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 205ebe4386f648e5967e82c48e602472: Generated new TSK 0
I20260925 00:47:26.718770 27978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:26.726714 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 27891
2026-09-25T00:47:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:26 dist-test-slave-w8nh krb5kdc[27990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:26 dist-test-slave-w8nh krb5kdc[27990](info): set up 2 sockets
Sep 25 00:47:26 dist-test-slave-w8nh krb5kdc[27990](info): commencing operation
krb5kdc: starting...
W20260925 00:47:27.772604 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:27 dist-test-slave-w8nh krb5kdc[27990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:47:27.900481 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44907
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:44907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:28.012379 28003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:28.012702 28003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:28.012782 28003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:28.016783 28003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:28.016883 28003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:28.016921 28003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:28.016981 28003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:28.017021 28003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:28.022282 28003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:44907
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:44907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:28.023393 28003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:28.024457 28003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:28.032927 28008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:28.033155 28003 server_base.cc:1034] running on GCE node
W20260925 00:47:28.033018 28011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:28.032979 28009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:28.033846 28003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:28.034893 28003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:28.036108 28003 hybrid_clock.cc:648] HybridClock initialized: now 1790297248036087 us; error 27 us; skew 500 ppm
Sep 25 00:47:28 dist-test-slave-w8nh krb5kdc[27990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297248, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:28.038995 28003 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:47:28.039963 28003 webserver.cc:460] Webserver started at http://127.6.72.126:46843/ using document root <none> and password file <none>
I20260925 00:47:28.040673 28003 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:28.040777 28003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:28.041023 28003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:28.043200 28003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "bb1fb31964bb4e90828097042e3f47de"
format_stamp: "Formatted at 2026-09-25 00:47:28 on dist-test-slave-w8nh"
server_key: "848dbc8bfd3d4beddcb21187fe3d9b8c"
server_key_iv: "8b5cf754f65597761c17398c9e6664a8"
server_key_version: "encryptionkey@0"
I20260925 00:47:28.043812 28003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "bb1fb31964bb4e90828097042e3f47de"
format_stamp: "Formatted at 2026-09-25 00:47:28 on dist-test-slave-w8nh"
server_key: "848dbc8bfd3d4beddcb21187fe3d9b8c"
server_key_iv: "8b5cf754f65597761c17398c9e6664a8"
server_key_version: "encryptionkey@0"
I20260925 00:47:28.047988 28003 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 00:47:28.050745 28018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:28.052068 28003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 00:47:28.052222 28003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "bb1fb31964bb4e90828097042e3f47de"
format_stamp: "Formatted at 2026-09-25 00:47:28 on dist-test-slave-w8nh"
server_key: "848dbc8bfd3d4beddcb21187fe3d9b8c"
server_key_iv: "8b5cf754f65597761c17398c9e6664a8"
server_key_version: "encryptionkey@0"
I20260925 00:47:28.052389 28003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:28.084728 28003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:28.087710 28003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:28.087918 28003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:28.095070 28003 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:44907
I20260925 00:47:28.095103 28070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:44907 every 8 connection(s)
I20260925 00:47:28.095988 28003 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:28.097175 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 28003
I20260925 00:47:28.097332 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:28.097622 6433 external_mini_cluster.cc:1428] Setting key aea796a1d71761c7f6983badd417b1a6
I20260925 00:47:28.099186 28071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:28.105844 28071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de: Bootstrap starting.
Sep 25 00:47:28 dist-test-slave-w8nh krb5kdc[27990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:28.108609 28071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:28.109545 28071 log.cc:824] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:28.112064 28071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de: No bootstrap required, opened a new log
I20260925 00:47:28.113415 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:28.099052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42835 (local address 127.6.72.126:44907)
0925 00:47:28.099513 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:28.099537 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:28.099579 (+ 42us) server_negotiation.cc:407] Connection header received
0925 00:47:28.100442 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:28.100464 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:28.100873 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:28.101232 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:28.101908 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:28.103374 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:28.104295 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:28.105004 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:28.105174 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:28.107915 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:28.107921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:28.107936 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:28.107969 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:28.110025 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:28.110647 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:28.110653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:28.110659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:28.110793 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:28.111123 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:28.111126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:28.111128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:28.111659 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:28.111863 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:28.112195 (+ 332us) server_negotiation.cc:299] Negotiation successful
0925 00:47:28.112490 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":154,"threads_started":1}
I20260925 00:47:28.116498 28071 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1fb31964bb4e90828097042e3f47de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44907 } }
I20260925 00:47:28.116765 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:28.116776 28071 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:28.116873 28071 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1fb31964bb4e90828097042e3f47de, State: Initialized, Role: FOLLOWER
I20260925 00:47:28.117297 28071 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1fb31964bb4e90828097042e3f47de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44907 } }
I20260925 00:47:28.117415 28071 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:28.117499 28071 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:28.117614 28071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:28.118657 28071 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1fb31964bb4e90828097042e3f47de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44907 } }
I20260925 00:47:28.119055 28071 leader_election.cc:302] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb1fb31964bb4e90828097042e3f47de; no voters:
I20260925 00:47:28.119443 28071 leader_election.cc:288] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:28.119544 28081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:28.119757 28081 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [term 1 LEADER]: Becoming Leader. State: Replica: bb1fb31964bb4e90828097042e3f47de, State: Running, Role: LEADER
I20260925 00:47:28.120092 28081 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1fb31964bb4e90828097042e3f47de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44907 } }
I20260925 00:47:28.120865 28071 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:28.121776 28082 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb1fb31964bb4e90828097042e3f47de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1fb31964bb4e90828097042e3f47de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44907 } } }
I20260925 00:47:28.121927 28082 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:28.122256 28083 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb1fb31964bb4e90828097042e3f47de. Latest consensus state: current_term: 1 leader_uuid: "bb1fb31964bb4e90828097042e3f47de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1fb31964bb4e90828097042e3f47de" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 44907 } } }
I20260925 00:47:28.122363 28083 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:28.123559 28090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:28.127091 28090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:28.128298 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:28.118965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51867 (local address 127.6.72.126:44907)
0925 00:47:28.119108 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:28.119112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:28.119260 (+ 148us) server_negotiation.cc:407] Connection header received
0925 00:47:28.119396 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:28.119399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:28.119442 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:28.119534 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:28.120420 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:28.121012 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:28.122212 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:28.122548 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:28.122629 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:28.124450 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:28.124454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:28.124457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:28.124484 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:28.126081 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:28.126793 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:28.126797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:28.126799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:28.126885 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:28.127277 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:28.127282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:28.127287 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:28.127511 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:28.127667 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:28.127992 (+ 325us) server_negotiation.cc:299] Negotiation successful
0925 00:47:28.128098 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 00:47:28.132627 28090 catalog_manager.cc:1347] Generated new cluster ID: 0bfef65518fe42de9a656f1358e05e3a
I20260925 00:47:28.132706 28090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:28.139213 28090 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:28.140086 28090 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:28.144476 28090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb1fb31964bb4e90828097042e3f47de: Generated new TSK 0
I20260925 00:47:28.145145 28090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:28.155416 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 28003
W20260925 00:47:28.160989 27988 connection.cc:535] client connection to 127.6.72.126:44907 recv error: Network error: failed to read from TLS socket (remote: 127.6.72.126:44907): Connection reset by peer (error 104)
2026-09-25T00:47:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:28 dist-test-slave-w8nh krb5kdc[28102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:28 dist-test-slave-w8nh krb5kdc[28102](info): set up 2 sockets
Sep 25 00:47:28 dist-test-slave-w8nh krb5kdc[28102](info): commencing operation
krb5kdc: starting...
W20260925 00:47:29.232991 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:29 dist-test-slave-w8nh krb5kdc[28102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:47:29.355566 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:42243
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:42243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:29.466104 28115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:29.466485 28115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:29.466568 28115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:29.470888 28115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:29.471019 28115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:29.471053 28115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:29.471091 28115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:29.471130 28115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:29.476868 28115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:42243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:42243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:29.478030 28115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:29.479087 28115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:29.487839 28123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:29.488005 28115 server_base.cc:1034] running on GCE node
W20260925 00:47:29.487860 28120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:29.487844 28121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:29.488798 28115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:29.489967 28115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:29.491128 28115 hybrid_clock.cc:648] HybridClock initialized: now 1790297249491117 us; error 24 us; skew 500 ppm
Sep 25 00:47:29 dist-test-slave-w8nh krb5kdc[28102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297249, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:29.493861 28115 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:47:29.494772 28115 webserver.cc:460] Webserver started at http://127.6.72.126:45189/ using document root <none> and password file <none>
I20260925 00:47:29.495327 28115 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:29.495386 28115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:29.495620 28115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:29.497730 28115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "f54eac174ad14366ac89a6959ff9b3ac"
format_stamp: "Formatted at 2026-09-25 00:47:29 on dist-test-slave-w8nh"
server_key: "912b5a161252322ea83054a6bda2b1c6"
server_key_iv: "281ed897734c85a44d0f3046e40868c6"
server_key_version: "encryptionkey@0"
I20260925 00:47:29.498301 28115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "f54eac174ad14366ac89a6959ff9b3ac"
format_stamp: "Formatted at 2026-09-25 00:47:29 on dist-test-slave-w8nh"
server_key: "912b5a161252322ea83054a6bda2b1c6"
server_key_iv: "281ed897734c85a44d0f3046e40868c6"
server_key_version: "encryptionkey@0"
I20260925 00:47:29.502128 28115 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:47:29.504681 28130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:29.506048 28115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 00:47:29.506213 28115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "f54eac174ad14366ac89a6959ff9b3ac"
format_stamp: "Formatted at 2026-09-25 00:47:29 on dist-test-slave-w8nh"
server_key: "912b5a161252322ea83054a6bda2b1c6"
server_key_iv: "281ed897734c85a44d0f3046e40868c6"
server_key_version: "encryptionkey@0"
I20260925 00:47:29.506325 28115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:29.521726 28115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:29.525036 28115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:29.525269 28115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:29.532812 28115 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:42243
I20260925 00:47:29.532841 28182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:42243 every 8 connection(s)
I20260925 00:47:29.533797 28115 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:29.536751 28183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:29.542428 28183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac: Bootstrap starting.
I20260925 00:47:29.542790 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 28115
I20260925 00:47:29.542923 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:29.543218 6433 external_mini_cluster.cc:1428] Setting key bb01703c38781804821a7e8c97889bec
I20260925 00:47:29.545303 28183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:29.546025 28183 log.cc:824] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:29.547865 28183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac: No bootstrap required, opened a new log
I20260925 00:47:29.551590 28183 raft_consensus.cc:348] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54eac174ad14366ac89a6959ff9b3ac" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42243 } }
I20260925 00:47:29.551836 28183 raft_consensus.cc:374] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:29.551918 28183 raft_consensus.cc:729] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f54eac174ad14366ac89a6959ff9b3ac, State: Initialized, Role: FOLLOWER
I20260925 00:47:29.552380 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:29.544579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55921 (local address 127.6.72.126:42243)
0925 00:47:29.545049 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:29.545064 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:29.545101 (+ 37us) server_negotiation.cc:407] Connection header received
0925 00:47:29.545840 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:29.545858 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:29.546252 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:29.546583 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:29.547000 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:29.547945 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:29.548911 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:29.549400 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:29.549575 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:29.550535 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:29.550540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:29.550553 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:29.550947 (+ 394us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:29.550999 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:29.551021 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:29.551164 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:29.551347 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 00:47:29.551485 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":152,"threads_started":1}
I20260925 00:47:29.552451 28183 consensus_queue.cc:260] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54eac174ad14366ac89a6959ff9b3ac" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42243 } }
I20260925 00:47:29.552642 28183 raft_consensus.cc:388] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:29.552717 28183 raft_consensus.cc:482] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:29.552817 28183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:29.553833 28183 raft_consensus.cc:504] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54eac174ad14366ac89a6959ff9b3ac" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42243 } }
I20260925 00:47:29.554212 28183 leader_election.cc:302] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f54eac174ad14366ac89a6959ff9b3ac; no voters:
I20260925 00:47:29.554559 28183 leader_election.cc:288] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:29.554720 28188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:29.554966 28188 raft_consensus.cc:686] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [term 1 LEADER]: Becoming Leader. State: Replica: f54eac174ad14366ac89a6959ff9b3ac, State: Running, Role: LEADER
I20260925 00:47:29.555495 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:29.555644 28188 consensus_queue.cc:237] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54eac174ad14366ac89a6959ff9b3ac" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42243 } }
I20260925 00:47:29.555900 28183 sys_catalog.cc:564] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:29.557363 28189 sys_catalog.cc:455] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f54eac174ad14366ac89a6959ff9b3ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54eac174ad14366ac89a6959ff9b3ac" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42243 } } }
I20260925 00:47:29.557484 28189 sys_catalog.cc:458] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:29.557834 28190 sys_catalog.cc:455] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader f54eac174ad14366ac89a6959ff9b3ac. Latest consensus state: current_term: 1 leader_uuid: "f54eac174ad14366ac89a6959ff9b3ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54eac174ad14366ac89a6959ff9b3ac" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 42243 } } }
I20260925 00:47:29.557917 28190 sys_catalog.cc:458] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:29.558482 28203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:29.561749 28203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:29.562777 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:29.557688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44107 (local address 127.6.72.126:42243)
0925 00:47:29.557823 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:29.557828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:29.558334 (+ 506us) server_negotiation.cc:407] Connection header received
0925 00:47:29.558515 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:29.558518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:29.558563 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:29.558636 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:29.558964 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:29.559518 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:29.560674 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:29.561005 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:29.561104 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:29.562161 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:29.562164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:29.562167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:29.562179 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:29.562206 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:29.562218 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:29.562302 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:29.562605 (+ 303us) server_negotiation.cc:299] Negotiation successful
0925 00:47:29.562643 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 00:47:29.567600 28203 catalog_manager.cc:1347] Generated new cluster ID: 39464c0c08334b4084091fd73befcfe6
I20260925 00:47:29.567672 28203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:29.575114 28203 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:29.576387 28203 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:29.586606 28203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f54eac174ad14366ac89a6959ff9b3ac: Generated new TSK 0
I20260925 00:47:29.587371 28203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:29.599922 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 28115
W20260925 00:47:29.605580 28100 connection.cc:535] client connection to 127.6.72.126:42243 recv error: Network error: failed to read from TLS socket (remote: 127.6.72.126:42243): Connection reset by peer (error 104)
2026-09-25T00:47:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1444 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:29 dist-test-slave-w8nh krb5kdc[28214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:29 dist-test-slave-w8nh krb5kdc[28214](info): set up 2 sockets
Sep 25 00:47:29 dist-test-slave-w8nh krb5kdc[28214](info): commencing operation
krb5kdc: starting...
W20260925 00:47:30.632592 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:30 dist-test-slave-w8nh krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:47:30.760654 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:32861
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:43673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:32861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:30.875516 28227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:30.875793 28227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:30.875835 28227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:30.879848 28227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:30.879912 28227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:30.879932 28227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:30.879949 28227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:30.879964 28227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:30.885228 28227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:43673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:32861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:32861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:30.886278 28227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:30.887327 28227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:30.895773 28233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:30.895774 28235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:30.895941 28227 server_base.cc:1034] running on GCE node
W20260925 00:47:30.895877 28232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:30.896628 28227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:30.897789 28227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:30.898978 28227 hybrid_clock.cc:648] HybridClock initialized: now 1790297250898943 us; error 46 us; skew 500 ppm
Sep 25 00:47:30 dist-test-slave-w8nh krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297250, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:30.901804 28227 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:47:30.902710 28227 webserver.cc:460] Webserver started at http://127.6.72.126:35007/ using document root <none> and password file <none>
I20260925 00:47:30.903285 28227 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:30.903379 28227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:30.903645 28227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:30.905773 28227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "2d2833564da745be85da56fa3ab1c416"
format_stamp: "Formatted at 2026-09-25 00:47:30 on dist-test-slave-w8nh"
server_key: "628402601afaef021940b0b3d4a3f967"
server_key_iv: "16673d5911eb9733aba83443bfcc2792"
server_key_version: "encryptionkey@0"
I20260925 00:47:30.906363 28227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "2d2833564da745be85da56fa3ab1c416"
format_stamp: "Formatted at 2026-09-25 00:47:30 on dist-test-slave-w8nh"
server_key: "628402601afaef021940b0b3d4a3f967"
server_key_iv: "16673d5911eb9733aba83443bfcc2792"
server_key_version: "encryptionkey@0"
I20260925 00:47:30.910293 28227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260925 00:47:30.912954 28242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:30.914397 28227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:30.914572 28227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "2d2833564da745be85da56fa3ab1c416"
format_stamp: "Formatted at 2026-09-25 00:47:30 on dist-test-slave-w8nh"
server_key: "628402601afaef021940b0b3d4a3f967"
server_key_iv: "16673d5911eb9733aba83443bfcc2792"
server_key_version: "encryptionkey@0"
I20260925 00:47:30.914719 28227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:30.919750 28227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:30.922791 28227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:30.922993 28227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:30.930334 28227 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:32861
I20260925 00:47:30.930373 28294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:32861 every 8 connection(s)
I20260925 00:47:30.931303 28227 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:30.934021 28295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:30.937444 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 28227
I20260925 00:47:30.937565 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:30.937876 6433 external_mini_cluster.cc:1428] Setting key 48ae284a30d0c528336a9a99fe89d34d
I20260925 00:47:30.940259 28295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416: Bootstrap starting.
I20260925 00:47:30.943075 28295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:30.943589 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:30.939326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57411 (local address 127.6.72.126:32861)
0925 00:47:30.939777 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:30.939795 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:30.939839 (+ 44us) server_negotiation.cc:407] Connection header received
0925 00:47:30.940701 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:30.940724 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:30.941084 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:30.941457 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:30.942172 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:30.942177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:30.942192 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:30.942203 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:30.942262 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:30.942287 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:30.942418 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:30.942486 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 00:47:30.942631 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":125,"threads_started":1}
I20260925 00:47:30.943957 28295 log.cc:824] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:30.946039 28295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416: No bootstrap required, opened a new log
I20260925 00:47:30.946499 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:30.950191 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:30.948565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59577 (local address 127.6.72.126:32861)
0925 00:47:30.948685 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:30.948689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:30.948844 (+ 155us) server_negotiation.cc:407] Connection header received
0925 00:47:30.948978 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:30.948980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:30.949008 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:30.949070 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:30.949799 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:30.949802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:30.949810 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:30.949820 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:30.949844 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:30.949854 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:30.949921 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:30.950025 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 00:47:30.950054 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 00:47:30.950460 28295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2833564da745be85da56fa3ab1c416" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 32861 } }
I20260925 00:47:30.950731 28295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:30.950788 28295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2833564da745be85da56fa3ab1c416, State: Initialized, Role: FOLLOWER
I20260925 00:47:30.951242 28295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2833564da745be85da56fa3ab1c416" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 32861 } }
I20260925 00:47:30.951398 28295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:30.951464 28295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:30.951558 28295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:30.952589 28295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2833564da745be85da56fa3ab1c416" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 32861 } }
I20260925 00:47:30.952939 28295 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d2833564da745be85da56fa3ab1c416; no voters:
I20260925 00:47:30.953240 28295 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:30.953358 28305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:30.953603 28305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [term 1 LEADER]: Becoming Leader. State: Replica: 2d2833564da745be85da56fa3ab1c416, State: Running, Role: LEADER
I20260925 00:47:30.953948 28305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2833564da745be85da56fa3ab1c416" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 32861 } }
I20260925 00:47:30.954514 28295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:30.955659 28307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d2833564da745be85da56fa3ab1c416. Latest consensus state: current_term: 1 leader_uuid: "2d2833564da745be85da56fa3ab1c416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2833564da745be85da56fa3ab1c416" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 32861 } } }
I20260925 00:47:30.955799 28307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:30.956064 28306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d2833564da745be85da56fa3ab1c416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2833564da745be85da56fa3ab1c416" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 32861 } } }
I20260925 00:47:30.956158 28306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416 [sys.catalog]: This master's current role is: LEADER
W20260925 00:47:30.959158 28320 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:30.959283 28320 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:30.959358 28314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:30.960039 28314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:30.965700 28314 catalog_manager.cc:1347] Generated new cluster ID: 50606e732fd8467ab4fccaf36f08d42b
I20260925 00:47:30.965785 28314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:30.987417 28314 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:30.987797 28314 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:30.993906 28314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d2833564da745be85da56fa3ab1c416: Generated new TSK 0
I20260925 00:47:30.994724 28314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:31.001490 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 28227
2026-09-25T00:47:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:31 dist-test-slave-w8nh krb5kdc[28326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:31 dist-test-slave-w8nh krb5kdc[28326](info): set up 2 sockets
Sep 25 00:47:31 dist-test-slave-w8nh krb5kdc[28326](info): commencing operation
krb5kdc: starting...
W20260925 00:47:32.060621 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:32 dist-test-slave-w8nh krb5kdc[28326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:47:32.187728 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:33557
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:46013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:33557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:32.307777 28339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:32.308094 28339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:32.308141 28339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:32.312438 28339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:32.312530 28339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:32.312554 28339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:32.312574 28339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:32.312598 28339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:32.318428 28339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:46013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:33557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:33557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:32.319571 28339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:32.320696 28339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:32.329615 28345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:32.329615 28344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:32.329675 28347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:32.330013 28339 server_base.cc:1034] running on GCE node
I20260925 00:47:32.330585 28339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:32.331725 28339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:32.332928 28339 hybrid_clock.cc:648] HybridClock initialized: now 1790297252332899 us; error 45 us; skew 500 ppm
Sep 25 00:47:32 dist-test-slave-w8nh krb5kdc[28326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297252, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:32.335904 28339 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:47:32.336942 28339 webserver.cc:460] Webserver started at http://127.6.72.126:42799/ using document root <none> and password file <none>
I20260925 00:47:32.337499 28339 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:32.337563 28339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:32.337746 28339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:32.339828 28339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "47466b50d7474fbda96aeedfd83eca17"
format_stamp: "Formatted at 2026-09-25 00:47:32 on dist-test-slave-w8nh"
server_key: "99880bcf5d1de6f59a99a9a0621d5813"
server_key_iv: "cff1bde7fd20d9cb68ce1d5351e5e8bc"
server_key_version: "encryptionkey@0"
I20260925 00:47:32.340441 28339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "47466b50d7474fbda96aeedfd83eca17"
format_stamp: "Formatted at 2026-09-25 00:47:32 on dist-test-slave-w8nh"
server_key: "99880bcf5d1de6f59a99a9a0621d5813"
server_key_iv: "cff1bde7fd20d9cb68ce1d5351e5e8bc"
server_key_version: "encryptionkey@0"
I20260925 00:47:32.344395 28339 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 00:47:32.346915 28354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:32.348193 28339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:47:32.348376 28339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "47466b50d7474fbda96aeedfd83eca17"
format_stamp: "Formatted at 2026-09-25 00:47:32 on dist-test-slave-w8nh"
server_key: "99880bcf5d1de6f59a99a9a0621d5813"
server_key_iv: "cff1bde7fd20d9cb68ce1d5351e5e8bc"
server_key_version: "encryptionkey@0"
I20260925 00:47:32.348480 28339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:32.365221 28339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:32.368655 28339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:32.368832 28339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:32.376343 28339 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:33557
I20260925 00:47:32.376359 28406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:33557 every 8 connection(s)
I20260925 00:47:32.377401 28339 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:32.380429 28407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:32.384866 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 28339
I20260925 00:47:32.385003 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:32.385329 6433 external_mini_cluster.cc:1428] Setting key b3a221e57737ccdfb0b3838a48377239
I20260925 00:47:32.386646 28407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17: Bootstrap starting.
I20260925 00:47:32.389097 28407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:32.390020 28407 log.cc:824] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:32.392681 28407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17: No bootstrap required, opened a new log
Sep 25 00:47:32 dist-test-slave-w8nh krb5kdc[28326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:32.398548 28407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47466b50d7474fbda96aeedfd83eca17" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33557 } }
I20260925 00:47:32.398903 28407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:32.398963 28407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47466b50d7474fbda96aeedfd83eca17, State: Initialized, Role: FOLLOWER
I20260925 00:47:32.399521 28407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47466b50d7474fbda96aeedfd83eca17" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33557 } }
I20260925 00:47:32.399721 28407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:32.399797 28407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:32.399935 28407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:32.401127 28407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47466b50d7474fbda96aeedfd83eca17" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33557 } }
I20260925 00:47:32.401561 28407 leader_election.cc:302] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47466b50d7474fbda96aeedfd83eca17; no voters:
I20260925 00:47:32.402010 28407 leader_election.cc:288] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:32.402108 28412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:32.402359 28412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [term 1 LEADER]: Becoming Leader. State: Replica: 47466b50d7474fbda96aeedfd83eca17, State: Running, Role: LEADER
I20260925 00:47:32.402405 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:32.386906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54451 (local address 127.6.72.126:33557)
0925 00:47:32.387341 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:32.387358 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:32.387393 (+ 35us) server_negotiation.cc:407] Connection header received
0925 00:47:32.388071 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:32.388088 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:32.388526 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:32.388835 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:32.389820 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:32.391103 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:32.392509 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:32.393072 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:32.393205 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:32.396462 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:32.396473 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:32.396502 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:32.396554 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:32.398974 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:32.399583 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:32.399590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:32.399606 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:32.399778 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:32.400130 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:32.400134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:32.400136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:32.400640 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:32.400871 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:32.401136 (+ 265us) server_negotiation.cc:299] Negotiation successful
0925 00:47:32.401465 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":148,"threads_started":1}
I20260925 00:47:32.402815 28412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47466b50d7474fbda96aeedfd83eca17" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33557 } }
I20260925 00:47:32.403561 28407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:32.404289 28412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47466b50d7474fbda96aeedfd83eca17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47466b50d7474fbda96aeedfd83eca17" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33557 } } }
I20260925 00:47:32.404497 28412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:32.405014 28419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:32.405941 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:32.408015 28413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47466b50d7474fbda96aeedfd83eca17. Latest consensus state: current_term: 1 leader_uuid: "47466b50d7474fbda96aeedfd83eca17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47466b50d7474fbda96aeedfd83eca17" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33557 } } }
I20260925 00:47:32.408140 28413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:32.409063 28419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:32.414974 28419 catalog_manager.cc:1347] Generated new cluster ID: f2847268cd5649c49ca2dff99165e964
I20260925 00:47:32.415076 28419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:32.423050 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:32.408670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43657 (local address 127.6.72.126:33557)
0925 00:47:32.408832 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:32.408837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:32.415459 (+ 6622us) server_negotiation.cc:407] Connection header received
0925 00:47:32.415646 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:32.415654 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:32.415698 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:32.415788 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:32.416558 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:32.417074 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:32.417911 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:32.418176 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:32.418297 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:32.419732 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:32.419737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:32.419740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:32.419769 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:32.421355 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:32.421793 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:32.421797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:32.421799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:32.421882 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:32.422190 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:32.422193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:32.422195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:32.422382 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:32.422517 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:32.422754 (+ 237us) server_negotiation.cc:299] Negotiation successful
0925 00:47:32.422865 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 00:47:32.431236 28419 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:32.432327 28419 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:32.439483 28419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47466b50d7474fbda96aeedfd83eca17: Generated new TSK 0
I20260925 00:47:32.440224 28419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:32.448400 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 28339
2026-09-25T00:47:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:32 dist-test-slave-w8nh krb5kdc[28438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:32 dist-test-slave-w8nh krb5kdc[28438](info): set up 2 sockets
Sep 25 00:47:32 dist-test-slave-w8nh krb5kdc[28438](info): commencing operation
krb5kdc: starting...
W20260925 00:47:33.476645 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:33 dist-test-slave-w8nh krb5kdc[28438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:47:33.603102 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:43587
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:40259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:43587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:33.715796 28451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:33.716076 28451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:33.716120 28451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:33.720499 28451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:33.720561 28451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:33.720582 28451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:33.720599 28451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:33.720645 28451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:33.726233 28451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:40259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:43587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:43587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:33.727355 28451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:33.728411 28451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:33.736954 28459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:33.736982 28457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:33.736982 28456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:33.737047 28451 server_base.cc:1034] running on GCE node
I20260925 00:47:33.737766 28451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:33.738857 28451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:33.740047 28451 hybrid_clock.cc:648] HybridClock initialized: now 1790297253740016 us; error 49 us; skew 500 ppm
Sep 25 00:47:33 dist-test-slave-w8nh krb5kdc[28438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297253, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:33.742848 28451 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:47:33.743764 28451 webserver.cc:460] Webserver started at http://127.6.72.126:40075/ using document root <none> and password file <none>
I20260925 00:47:33.744381 28451 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:33.744472 28451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:33.744681 28451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:33.746727 28451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "5305ab9e7dbc4214b2a53e140275828c"
format_stamp: "Formatted at 2026-09-25 00:47:33 on dist-test-slave-w8nh"
server_key: "1287f8a174e94f3578f5aca97eb765f6"
server_key_iv: "25ed0d8890f2a05bd56cf73f89496025"
server_key_version: "encryptionkey@0"
I20260925 00:47:33.747295 28451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "5305ab9e7dbc4214b2a53e140275828c"
format_stamp: "Formatted at 2026-09-25 00:47:33 on dist-test-slave-w8nh"
server_key: "1287f8a174e94f3578f5aca97eb765f6"
server_key_iv: "25ed0d8890f2a05bd56cf73f89496025"
server_key_version: "encryptionkey@0"
I20260925 00:47:33.751137 28451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260925 00:47:33.753713 28466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:33.754995 28451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:47:33.755136 28451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "5305ab9e7dbc4214b2a53e140275828c"
format_stamp: "Formatted at 2026-09-25 00:47:33 on dist-test-slave-w8nh"
server_key: "1287f8a174e94f3578f5aca97eb765f6"
server_key_iv: "25ed0d8890f2a05bd56cf73f89496025"
server_key_version: "encryptionkey@0"
I20260925 00:47:33.755251 28451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:33.771863 28451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:33.775166 28451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:33.775375 28451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:33.782582 28451 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:43587
I20260925 00:47:33.782600 28518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:43587 every 8 connection(s)
I20260925 00:47:33.783496 28451 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:33.786192 28519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:33.790043 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 28451
I20260925 00:47:33.790197 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:33.790482 6433 external_mini_cluster.cc:1428] Setting key 38add28b5ec3651f52df8683549d4fdc
I20260925 00:47:33.791980 28519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c: Bootstrap starting.
I20260925 00:47:33.794338 28519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:33.795091 28519 log.cc:824] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:33.797338 28519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c: No bootstrap required, opened a new log
I20260925 00:47:33.799396 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:33.791883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35921 (local address 127.6.72.126:43587)
0925 00:47:33.792307 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:33.792323 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:33.792357 (+ 34us) server_negotiation.cc:407] Connection header received
0925 00:47:33.792974 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:33.792990 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:33.793317 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:33.793588 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:33.793961 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:33.794989 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:33.796068 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:33.796569 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:33.796710 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:33.797547 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:33.797552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:33.797565 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:33.797945 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:33.797995 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:33.798017 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:33.798092 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:33.798261 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 00:47:33.798502 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":126,"threads_started":1}
I20260925 00:47:33.800716 28519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5305ab9e7dbc4214b2a53e140275828c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43587 } }
I20260925 00:47:33.800954 28519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:33.801030 28519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5305ab9e7dbc4214b2a53e140275828c, State: Initialized, Role: FOLLOWER
I20260925 00:47:33.801517 28519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5305ab9e7dbc4214b2a53e140275828c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43587 } }
I20260925 00:47:33.801707 28519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:33.801793 28519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:33.801893 28519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:33.802460 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:33.802920 28519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5305ab9e7dbc4214b2a53e140275828c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43587 } }
I20260925 00:47:33.803251 28519 leader_election.cc:302] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5305ab9e7dbc4214b2a53e140275828c; no voters:
I20260925 00:47:33.803540 28519 leader_election.cc:288] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:33.803656 28524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:33.803934 28524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [term 1 LEADER]: Becoming Leader. State: Replica: 5305ab9e7dbc4214b2a53e140275828c, State: Running, Role: LEADER
I20260925 00:47:33.804327 28524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5305ab9e7dbc4214b2a53e140275828c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43587 } }
I20260925 00:47:33.804904 28519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:33.806054 28528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5305ab9e7dbc4214b2a53e140275828c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5305ab9e7dbc4214b2a53e140275828c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43587 } } }
I20260925 00:47:33.806246 28528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:33.806494 28528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5305ab9e7dbc4214b2a53e140275828c. Latest consensus state: current_term: 1 leader_uuid: "5305ab9e7dbc4214b2a53e140275828c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5305ab9e7dbc4214b2a53e140275828c" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 43587 } } }
I20260925 00:47:33.806641 28528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:33.807505 28538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:33.809235 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:33.804587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50713 (local address 127.6.72.126:43587)
0925 00:47:33.804690 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:33.804694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:33.804992 (+ 298us) server_negotiation.cc:407] Connection header received
0925 00:47:33.805172 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:33.805175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:33.805224 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:33.805317 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:33.805635 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:33.806442 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:33.807573 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:33.807898 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:33.807981 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:33.808808 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:33.808811 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:33.808813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:33.808823 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:33.808846 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:33.808855 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:33.808951 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:33.809045 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 00:47:33.809095 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 00:47:33.810765 28538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:33.816701 28538 catalog_manager.cc:1347] Generated new cluster ID: 3a198e783fef4339881ef641e7904a4b
I20260925 00:47:33.816776 28538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:33.834395 28538 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:33.835345 28538 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:33.840878 28538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5305ab9e7dbc4214b2a53e140275828c: Generated new TSK 0
I20260925 00:47:33.841652 28538 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:33.851558 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 28451
2026-09-25T00:47:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:33 dist-test-slave-w8nh krb5kdc[28550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:33 dist-test-slave-w8nh krb5kdc[28550](info): set up 2 sockets
Sep 25 00:47:33 dist-test-slave-w8nh krb5kdc[28550](info): commencing operation
krb5kdc: starting...
W20260925 00:47:34.896556 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:34 dist-test-slave-w8nh krb5kdc[28550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:47:35.029016 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:40063
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:35863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:40063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:35.153026 28563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:35.153375 28563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:35.153473 28563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:35.157863 28563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:35.157971 28563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:35.158008 28563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:35.158061 28563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:35.158100 28563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:35.163729 28563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:35863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:40063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:40063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:35.165045 28563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:35.166107 28563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:35.174926 28568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:35.174952 28569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:35.174993 28571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:35.175213 28563 server_base.cc:1034] running on GCE node
I20260925 00:47:35.175804 28563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:35.177201 28563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:35.178439 28563 hybrid_clock.cc:648] HybridClock initialized: now 1790297255178408 us; error 51 us; skew 500 ppm
Sep 25 00:47:35 dist-test-slave-w8nh krb5kdc[28550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297255, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:35.181798 28563 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:47:35.182801 28563 webserver.cc:460] Webserver started at http://127.6.72.126:44563/ using document root <none> and password file <none>
I20260925 00:47:35.183416 28563 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:35.183518 28563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:35.183776 28563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:35.186179 28563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "14985d18a0e64f2d9bc01504f5ba195e"
format_stamp: "Formatted at 2026-09-25 00:47:35 on dist-test-slave-w8nh"
server_key: "2ae506b19c5e95f81dcd488162a97eb6"
server_key_iv: "5ecd1761d4aad475b05d0f3e3caceaec"
server_key_version: "encryptionkey@0"
I20260925 00:47:35.186817 28563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "14985d18a0e64f2d9bc01504f5ba195e"
format_stamp: "Formatted at 2026-09-25 00:47:35 on dist-test-slave-w8nh"
server_key: "2ae506b19c5e95f81dcd488162a97eb6"
server_key_iv: "5ecd1761d4aad475b05d0f3e3caceaec"
server_key_version: "encryptionkey@0"
I20260925 00:47:35.191026 28563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:47:35.193801 28578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:35.195284 28563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 00:47:35.195454 28563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "14985d18a0e64f2d9bc01504f5ba195e"
format_stamp: "Formatted at 2026-09-25 00:47:35 on dist-test-slave-w8nh"
server_key: "2ae506b19c5e95f81dcd488162a97eb6"
server_key_iv: "5ecd1761d4aad475b05d0f3e3caceaec"
server_key_version: "encryptionkey@0"
I20260925 00:47:35.195577 28563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:35.200150 28563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:35.203504 28563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:35.203706 28563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:35.211057 28563 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:40063
I20260925 00:47:35.211112 28630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:40063 every 8 connection(s)
I20260925 00:47:35.212064 28563 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:35.215104 28631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:35.215802 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 28563
I20260925 00:47:35.215965 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:35.216323 6433 external_mini_cluster.cc:1428] Setting key 00cf2c9bb674bfd237e762ab4883549c
I20260925 00:47:35.221935 28631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e: Bootstrap starting.
I20260925 00:47:35.221995 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:35.217725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37581 (local address 127.6.72.126:40063)
0925 00:47:35.218155 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:35.218174 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:35.218217 (+ 43us) server_negotiation.cc:407] Connection header received
0925 00:47:35.219016 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:35.219039 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:35.219387 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:35.219756 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:35.220543 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:35.220549 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:35.220565 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:35.220577 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:35.220642 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:35.220672 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:35.220830 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:35.220961 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 00:47:35.221114 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":99,"threads_started":1}
I20260925 00:47:35.224236 28631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:35.225006 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:35.225065 28631 log.cc:824] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:35.227193 28631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e: No bootstrap required, opened a new log
I20260925 00:47:35.229154 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:35.227220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51623 (local address 127.6.72.126:40063)
0925 00:47:35.227384 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:35.227387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:35.227606 (+ 219us) server_negotiation.cc:407] Connection header received
0925 00:47:35.227722 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:35.227724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:35.227755 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:35.227829 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:35.228653 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:35.228655 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:35.228657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 00:47:35.228676 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:35.228700 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 00:47:35.228711 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:35.228782 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:35.228884 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 00:47:35.228915 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260925 00:47:35.231459 28631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14985d18a0e64f2d9bc01504f5ba195e" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40063 } }
I20260925 00:47:35.231710 28631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 00:47:35.231799 28631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14985d18a0e64f2d9bc01504f5ba195e, State: Initialized, Role: FOLLOWER
I20260925 00:47:35.232275 28631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14985d18a0e64f2d9bc01504f5ba195e" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40063 } }
I20260925 00:47:35.232435 28631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 00:47:35.232506 28631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 00:47:35.232604 28631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:35.233531 28631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14985d18a0e64f2d9bc01504f5ba195e" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40063 } }
I20260925 00:47:35.233875 28631 leader_election.cc:302] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14985d18a0e64f2d9bc01504f5ba195e; no voters:
I20260925 00:47:35.234192 28631 leader_election.cc:288] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 00:47:35.234362 28641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:35.234577 28641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [term 1 LEADER]: Becoming Leader. State: Replica: 14985d18a0e64f2d9bc01504f5ba195e, State: Running, Role: LEADER
I20260925 00:47:35.234941 28641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14985d18a0e64f2d9bc01504f5ba195e" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40063 } }
I20260925 00:47:35.235455 28631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:35.236722 28642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14985d18a0e64f2d9bc01504f5ba195e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14985d18a0e64f2d9bc01504f5ba195e" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40063 } } }
I20260925 00:47:35.236860 28642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:35.237141 28643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14985d18a0e64f2d9bc01504f5ba195e. Latest consensus state: current_term: 1 leader_uuid: "14985d18a0e64f2d9bc01504f5ba195e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14985d18a0e64f2d9bc01504f5ba195e" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 40063 } } }
I20260925 00:47:35.237237 28643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:35.237319 28651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:35.240018 28651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:35.245777 28651 catalog_manager.cc:1347] Generated new cluster ID: 9a700ba32ee949608fe7c8a9b54df538
I20260925 00:47:35.245883 28651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:35.265666 28651 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:35.266033 28651 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:35.274397 28651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14985d18a0e64f2d9bc01504f5ba195e: Generated new TSK 0
I20260925 00:47:35.275265 28651 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 00:47:35.284472 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 28563
2026-09-25T00:47:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1432 ms)
[----------] 18 tests from AuthTokenIssuingTest (25951 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:35 dist-test-slave-w8nh krb5kdc[28662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:35 dist-test-slave-w8nh krb5kdc[28662](info): set up 2 sockets
Sep 25 00:47:35 dist-test-slave-w8nh krb5kdc[28662](info): commencing operation
krb5kdc: starting...
W20260925 00:47:36.336702 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:36 dist-test-slave-w8nh krb5kdc[28662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:47:36.470628 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:33683
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:33683,127.6.72.125:40687,127.6.72.124:37795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:36.588847 28675 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 00:47:36.589180 28675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:36.589208 28675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:36.589320 28675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:36.594092 28675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:36.594178 28675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:36.594201 28675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:36.594220 28675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:36.594238 28675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:36.600558 28675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:33683,127.6.72.125:40687,127.6.72.124:37795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:33683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:36.601774 28675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:36.602852 28675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:36.611819 28683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:36.611899 28675 server_base.cc:1034] running on GCE node
W20260925 00:47:36.611819 28681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:36.612135 28680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:36.612810 28675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:36.613969 28675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:36.615209 28675 hybrid_clock.cc:648] HybridClock initialized: now 1790297256615179 us; error 41 us; skew 500 ppm
Sep 25 00:47:36 dist-test-slave-w8nh krb5kdc[28662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:36.618181 28675 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:47:36.619122 28675 webserver.cc:460] Webserver started at http://127.6.72.126:44429/ using document root <none> and password file <none>
I20260925 00:47:36.619693 28675 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:36.619763 28675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:36.619988 28675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:36.622155 28675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "a9a90ccf65984e8ea42b1e899cb5a234"
format_stamp: "Formatted at 2026-09-25 00:47:36 on dist-test-slave-w8nh"
server_key: "1cfb46f92fc937da6087a1faf288ff05"
server_key_iv: "7b21e77bac3d8a4bb1f340e85a58ed5f"
server_key_version: "encryptionkey@0"
I20260925 00:47:36.622748 28675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "a9a90ccf65984e8ea42b1e899cb5a234"
format_stamp: "Formatted at 2026-09-25 00:47:36 on dist-test-slave-w8nh"
server_key: "1cfb46f92fc937da6087a1faf288ff05"
server_key_iv: "7b21e77bac3d8a4bb1f340e85a58ed5f"
server_key_version: "encryptionkey@0"
I20260925 00:47:36.626754 28675 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:47:36.629348 28690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:36.630698 28675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:36.630846 28675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "a9a90ccf65984e8ea42b1e899cb5a234"
format_stamp: "Formatted at 2026-09-25 00:47:36 on dist-test-slave-w8nh"
server_key: "1cfb46f92fc937da6087a1faf288ff05"
server_key_iv: "7b21e77bac3d8a4bb1f340e85a58ed5f"
server_key_version: "encryptionkey@0"
I20260925 00:47:36.630972 28675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:36.648504 28675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:36.651794 28675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:36.652021 28675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:36.659543 28675 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:33683
I20260925 00:47:36.659854 28742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:33683 every 8 connection(s)
I20260925 00:47:36.660650 28675 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:36.663643 28743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:36.667975 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 28675
I20260925 00:47:36.668118 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:36.667896 28743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:36.668447 6433 external_mini_cluster.cc:1428] Setting key 36d16cd305e31df04aad8bd0d8a2d52f
WARNING: no policy specified for kudu/127.6.72.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.125@KRBTEST.COM" created.
Sep 25 00:47:36 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:36.685918 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.670720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:33683 (local address 127.0.0.1:58014)
0925 00:47:36.671324 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.671407 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:36.672409 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:36.673153 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:36.673159 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:36.673627 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:36.673874 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:36.673888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.675072 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.675074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:36.675806 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:36.675815 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.676587 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.676591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:36.676683 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:36.677229 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:36.677246 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:36.680401 (+ 3155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:36.682910 (+ 2509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:36.682918 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:36.682928 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:36.683162 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:36.683539 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:36.683541 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:36.683543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:36.683650 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:36.684502 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:36.684507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:36.684639 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 00:47:36.684787 (+ 148us) client_negotiation.cc:240] Negotiation successful
0925 00:47:36.685071 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":148,"threads_started":1}
I20260925 00:47:36.685927 28747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.671619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58014 (local address 127.6.72.126:33683)
0925 00:47:36.671919 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:36.671930 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:36.671977 (+ 47us) server_negotiation.cc:407] Connection header received
0925 00:47:36.672724 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:36.672732 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:36.672960 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:36.673159 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:36.674028 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.674920 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.675927 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.676422 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.676590 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:36.680609 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:36.680616 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:36.680628 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:36.680653 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:36.682783 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:36.683279 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:36.683288 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:36.683293 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:36.683440 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:36.683753 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:36.683759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:36.683764 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:36.684385 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:36.684548 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:36.684921 (+ 373us) server_negotiation.cc:299] Negotiation successful
0925 00:47:36.685188 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":194,"thread_start_us":94,"threads_started":1}
I20260925 00:47:36.688723 28743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:36.689559 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.689233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:40687 (local address 127.0.0.1:40620)
0925 00:47:36.689419 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.689491 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260925 00:47:36.689800 28692 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.72.125:40687: connect: Connection refused (error 111)
I20260925 00:47:36.690780 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.690467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:40687 (local address 127.0.0.1:40626)
0925 00:47:36.690665 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.690719 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260925 00:47:36.690958 28743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.125:40687: Network error: Client connection negotiation failed: client connection to 127.6.72.125:40687: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab.
Entry for principal kudu/127.6.72.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab.
I20260925 00:47:36.699105 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.125:40687
--webserver_interface=127.6.72.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab
--principal=kudu/127.6.72.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:33683,127.6.72.125:40687,127.6.72.124:37795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:47:36.740218 28743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } attempt: 1
I20260925 00:47:36.741139 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.740831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:40687 (local address 127.0.0.1:40634)
0925 00:47:36.740994 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.741065 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 00:47:36.741873 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.741642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:40687 (local address 127.0.0.1:40648)
0925 00:47:36.741769 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.741812 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 00:47:36.742002 28743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.125:40687: Network error: Client connection negotiation failed: client connection to 127.6.72.125:40687: connect: Connection refused (error 111)
I20260925 00:47:36.810164 28743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } attempt: 2
I20260925 00:47:36.810988 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.810703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:40687 (local address 127.0.0.1:40664)
0925 00:47:36.810853 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.810914 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 00:47:36.811709 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.811496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:40687 (local address 127.0.0.1:40676)
0925 00:47:36.811601 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.811650 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 00:47:36.811820 28743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.125:40687: Network error: Client connection negotiation failed: client connection to 127.6.72.125:40687: connect: Connection refused (error 111)
W20260925 00:47:36.812592 28750 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 00:47:36.812909 28750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:36.812969 28750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:36.813046 28750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:36.817137 28750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:36.817217 28750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:36.817263 28750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:36.817317 28750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:36.817351 28750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:36.822904 28750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:33683,127.6.72.125:40687,127.6.72.124:37795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab
--principal=kudu/127.6.72.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.125:40687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.72.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:36.824070 28750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:36.825181 28750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:36.833657 28756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:36.833657 28755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:36.833844 28750 server_base.cc:1034] running on GCE node
W20260925 00:47:36.833717 28758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:36.834569 28750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:36.835779 28750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:36.836947 28750 hybrid_clock.cc:648] HybridClock initialized: now 1790297256836935 us; error 29 us; skew 500 ppm
Sep 25 00:47:36 dist-test-slave-w8nh krb5kdc[28662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:36.839700 28750 init.cc:377] Logged in from keytab as kudu/127.6.72.125@KRBTEST.COM (short username kudu)
I20260925 00:47:36.840731 28750 webserver.cc:460] Webserver started at http://127.6.72.125:36253/ using document root <none> and password file <none>
I20260925 00:47:36.841310 28750 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:36.841403 28750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:36.841629 28750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:36.843699 28750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/data/instance:
uuid: "4ae27fd311d0415093c6fef3fb3501d6"
format_stamp: "Formatted at 2026-09-25 00:47:36 on dist-test-slave-w8nh"
server_key: "ffc584f98dc7872d779fc3820601fb53"
server_key_iv: "abb03d068dbdc9f0746403f335a71e7d"
server_key_version: "encryptionkey@0"
I20260925 00:47:36.844326 28750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/wal/instance:
uuid: "4ae27fd311d0415093c6fef3fb3501d6"
format_stamp: "Formatted at 2026-09-25 00:47:36 on dist-test-slave-w8nh"
server_key: "ffc584f98dc7872d779fc3820601fb53"
server_key_iv: "abb03d068dbdc9f0746403f335a71e7d"
server_key_version: "encryptionkey@0"
I20260925 00:47:36.848059 28750 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 00:47:36.850579 28765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:36.851809 28750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 00:47:36.851959 28750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/wal
uuid: "4ae27fd311d0415093c6fef3fb3501d6"
format_stamp: "Formatted at 2026-09-25 00:47:36 on dist-test-slave-w8nh"
server_key: "ffc584f98dc7872d779fc3820601fb53"
server_key_iv: "abb03d068dbdc9f0746403f335a71e7d"
server_key_version: "encryptionkey@0"
I20260925 00:47:36.852073 28750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:36.866784 28750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:36.870059 28750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:36.870296 28750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:36.877700 28750 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.125:40687
I20260925 00:47:36.878013 28817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.125:40687 every 8 connection(s)
I20260925 00:47:36.878778 28750 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
I20260925 00:47:36.881799 28818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:36.885859 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 28750
I20260925 00:47:36.886019 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-1/wal/instance
I20260925 00:47:36.885756 28818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:36.886333 6433 external_mini_cluster.cc:1428] Setting key d5efaed3a7edad075db5e9a82c2bd179
WARNING: no policy specified for kudu/127.6.72.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.124@KRBTEST.COM" created.
Sep 25 00:47:36 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:36.903498 28743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } attempt: 3
I20260925 00:47:36.906740 28747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.887873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58028 (local address 127.6.72.126:33683)
0925 00:47:36.888074 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:36.888081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:36.889110 (+ 1029us) server_negotiation.cc:407] Connection header received
0925 00:47:36.890574 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:36.890578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:36.890654 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:36.890732 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:36.892484 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.893015 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.894825 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.895056 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.895182 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:36.901374 (+ 6192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:36.901377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:36.901380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:36.901408 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:36.903327 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:36.904290 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:36.904297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:36.904301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:36.904404 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:36.904907 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:36.904912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:36.904913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:36.905181 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:36.905284 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:36.906459 (+ 1175us) server_negotiation.cc:299] Negotiation successful
0925 00:47:36.906591 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 00:47:36.907289 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.888353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:33683 (local address 127.0.0.1:58028)
0925 00:47:36.888918 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.888980 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:36.890301 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:36.890973 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:36.890985 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:36.891506 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:36.892317 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:36.892342 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.893158 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.893163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:36.894673 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:36.894688 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.895227 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.895231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:36.895405 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:36.899090 (+ 3685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:36.899110 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:36.901117 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:36.903520 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:36.903560 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:36.903584 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:36.904101 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:36.904533 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:36.904536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:36.904538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:36.904772 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:36.905296 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:36.905304 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:36.905556 (+ 252us) client_negotiation.cc:769] Sending connection context
0925 00:47:36.905757 (+ 201us) client_negotiation.cc:240] Negotiation successful
0925 00:47:36.906145 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":128,"threads_started":1}
I20260925 00:47:36.908907 28818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 00:47:36 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
Sep 25 00:47:36 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
I20260925 00:47:36.914968 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.903992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:40687 (local address 127.0.0.1:40682)
0925 00:47:36.904191 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.904205 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:36.904329 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:36.905061 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:36.905065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:36.905271 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:36.905416 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:36.905427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.906983 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.906987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:36.907771 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:36.907780 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.908593 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.908601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:36.908776 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:36.909400 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:36.909411 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:36.910951 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:36.913204 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:36.913207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:36.913209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:36.913419 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:36.913751 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:36.913754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:36.913756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:36.913848 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:36.914354 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:36.914357 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:36.914482 (+ 125us) client_negotiation.cc:769] Sending connection context
0925 00:47:36.914627 (+ 145us) client_negotiation.cc:240] Negotiation successful
0925 00:47:36.914778 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab.
Entry for principal kudu/127.6.72.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab.
I20260925 00:47:36.915910 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.904169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.6.72.125:40687)
0925 00:47:36.904608 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:36.904619 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:36.904642 (+ 23us) server_negotiation.cc:407] Connection header received
0925 00:47:36.904687 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:36.904692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:36.904907 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:36.905054 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:36.905562 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.906830 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.907919 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.908440 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.908595 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:36.911103 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:36.911116 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:36.911132 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:36.911158 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:36.913053 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:36.913543 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:36.913551 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:36.913556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:36.913653 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:36.913959 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:36.913962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:36.913964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:36.914248 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:36.914391 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:36.914727 (+ 336us) server_negotiation.cc:299] Negotiation successful
0925 00:47:36.914907 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":195,"threads_started":1}
I20260925 00:47:36.916142 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.124:37795
--webserver_interface=127.6.72.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab
--principal=kudu/127.6.72.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:33683,127.6.72.125:40687,127.6.72.124:37795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:47:36.917986 28743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:36.918709 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.918468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58574)
0925 00:47:36.918583 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.918635 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 00:47:36.918756 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.909700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.6.72.125:40687)
0925 00:47:36.910067 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:36.910077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:36.910094 (+ 17us) server_negotiation.cc:407] Connection header received
0925 00:47:36.910133 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:36.910137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:36.910191 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:36.910283 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:36.910809 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.911330 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.912502 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.912748 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.912858 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:36.914819 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:36.914823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:36.914825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:36.914844 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:36.916879 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:36.917266 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:36.917272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:36.917274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:36.917354 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:36.917658 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:36.917661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:36.917662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:36.917902 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:36.917978 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:36.918314 (+ 336us) server_negotiation.cc:299] Negotiation successful
0925 00:47:36.918440 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":299,"thread_start_us":149,"threads_started":1}
I20260925 00:47:36.919209 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.909626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:40687 (local address 127.0.0.1:40686)
0925 00:47:36.909808 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.909825 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:36.909904 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:36.910330 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:36.910333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:36.910510 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:36.910639 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:36.910649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.911681 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.911683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:36.912371 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:36.912379 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:36.912855 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:36.912859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:36.912978 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:36.913416 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:36.913429 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:36.914685 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:36.916977 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:36.916980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:36.916981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:36.917170 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:36.917448 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:36.917451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:36.917453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:36.917562 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:36.918044 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:36.918048 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:36.918136 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 00:47:36.918887 (+ 751us) client_negotiation.cc:240] Negotiation successful
0925 00:47:36.919027 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":19}
I20260925 00:47:36.919775 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.919220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58578)
0925 00:47:36.919592 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.919653 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":112,"threads_started":1}
W20260925 00:47:36.920063 28743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:37795: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
I20260925 00:47:36.920095 28818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:36.920842 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.920541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58588)
0925 00:47:36.920672 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.920761 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 00:47:36.921053 28766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
I20260925 00:47:36.921984 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.921708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58594)
0925 00:47:36.921844 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.921906 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 00:47:36.922216 28818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:37795: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
I20260925 00:47:36.951267 28743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } attempt: 1
I20260925 00:47:36.952260 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.951870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58596)
0925 00:47:36.952055 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.952134 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 00:47:36.953112 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.952860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58606)
0925 00:47:36.952984 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.953044 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 00:47:36.953308 28743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:37795: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
I20260925 00:47:36.971519 28818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } attempt: 1
I20260925 00:47:36.972443 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.972081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58622)
0925 00:47:36.972279 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.972350 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260925 00:47:36.973156 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:36.972934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58632)
0925 00:47:36.973035 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:36.973085 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 00:47:36.973318 28818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:37795: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
I20260925 00:47:37.028478 28743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } attempt: 2
I20260925 00:47:37.029358 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.029008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58642)
0925 00:47:37.029185 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.029259 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 00:47:37.030156 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.029920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58656)
0925 00:47:37.030032 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.030082 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 00:47:37.030287 28743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:37795: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
W20260925 00:47:37.031780 28825 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 00:47:37.032038 28825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:37.032068 28825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:37.032109 28825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:37.036708 28825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:37.036767 28825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:37.036787 28825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:37.036803 28825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:37.036829 28825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:37.041472 28818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } attempt: 2
I20260925 00:47:37.042279 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.041958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58664)
0925 00:47:37.042119 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.042194 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 00:47:37.042718 28825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:33683,127.6.72.125:40687,127.6.72.124:37795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab
--principal=kudu/127.6.72.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.124:37795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.72.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:37.042969 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.042728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58680)
0925 00:47:37.042858 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.042908 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 00:47:37.043083 28818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:37795: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
I20260925 00:47:37.043896 28825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:37.044972 28825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:37.053093 28832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:37.053128 28833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:37.053263 28835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:37.053431 28825 server_base.cc:1034] running on GCE node
I20260925 00:47:37.054050 28825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:37.055099 28825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:37.056325 28825 hybrid_clock.cc:648] HybridClock initialized: now 1790297257056213 us; error 117 us; skew 500 ppm
Sep 25 00:47:37 dist-test-slave-w8nh krb5kdc[28662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297257, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:37.059165 28825 init.cc:377] Logged in from keytab as kudu/127.6.72.124@KRBTEST.COM (short username kudu)
I20260925 00:47:37.060036 28825 webserver.cc:460] Webserver started at http://127.6.72.124:36105/ using document root <none> and password file <none>
I20260925 00:47:37.060663 28825 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:37.060766 28825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:37.060998 28825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:37.063032 28825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/data/instance:
uuid: "36dc5cb75c3b4f88b35d531e3b253f15"
format_stamp: "Formatted at 2026-09-25 00:47:37 on dist-test-slave-w8nh"
server_key: "79e730fcb9d7c0e9932e25db37748b9e"
server_key_iv: "fa859ce338af4a166c253b388c2ca252"
server_key_version: "encryptionkey@0"
I20260925 00:47:37.063669 28825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/wal/instance:
uuid: "36dc5cb75c3b4f88b35d531e3b253f15"
format_stamp: "Formatted at 2026-09-25 00:47:37 on dist-test-slave-w8nh"
server_key: "79e730fcb9d7c0e9932e25db37748b9e"
server_key_iv: "fa859ce338af4a166c253b388c2ca252"
server_key_version: "encryptionkey@0"
I20260925 00:47:37.067337 28825 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260925 00:47:37.069808 28842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:37.070933 28825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:47:37.071072 28825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/wal
uuid: "36dc5cb75c3b4f88b35d531e3b253f15"
format_stamp: "Formatted at 2026-09-25 00:47:37 on dist-test-slave-w8nh"
server_key: "79e730fcb9d7c0e9932e25db37748b9e"
server_key_iv: "fa859ce338af4a166c253b388c2ca252"
server_key_version: "encryptionkey@0"
I20260925 00:47:37.071178 28825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:37.119824 28825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:37.123291 28825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:37.123510 28825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:37.129491 28743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } attempt: 3
I20260925 00:47:37.130399 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.130014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58692)
0925 00:47:37.130211 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.130300 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260925 00:47:37.131448 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.131190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58704)
0925 00:47:37.131322 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.131381 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260925 00:47:37.131644 28743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:37795: Network error: Client connection negotiation failed: client connection to 127.6.72.124:37795: connect: Connection refused (error 111)
I20260925 00:47:37.132477 28825 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.124:37795
I20260925 00:47:37.132790 28894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.124:37795 every 8 connection(s)
I20260925 00:47:37.133796 28825 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
I20260925 00:47:37.134256 28818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } attempt: 3
I20260925 00:47:37.134479 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 28825
I20260925 00:47:37.134630 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790296971355272-6433-0/minicluster-data/master-2/wal/instance
I20260925 00:47:37.134920 6433 external_mini_cluster.cc:1428] Setting key 53cd1ad693fdeac3b9040ff11d5ea1b4
I20260925 00:47:37.137087 28895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:37.141068 28895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 00:47:37 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
Sep 25 00:47:37 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
Sep 25 00:47:37 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297257, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:37.149880 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.134732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58716)
0925 00:47:37.134856 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.134871 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:37.134985 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:37.138361 (+ 3376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:37.138365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:37.138557 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:37.138678 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:37.138687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.140231 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.140264 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:37.141113 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:37.141126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.142133 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.142136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:37.142218 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.142819 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:37.142834 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:37.144899 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:37.147690 (+ 2791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:37.147696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:37.147701 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:37.147988 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:37.148347 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:37.148351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:37.148352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:37.148442 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:37.149286 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:37.149291 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:37.149398 (+ 107us) client_negotiation.cc:769] Sending connection context
0925 00:47:37.149521 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 00:47:37.149661 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 00:47:37.150972 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.135916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.6.72.124:37795)
0925 00:47:37.136885 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.136893 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.136904 (+ 11us) server_negotiation.cc:407] Connection header received
0925 00:47:37.137539 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.137562 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.138022 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.138384 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.138874 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.140079 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.141244 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.142009 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.142149 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.145073 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.145079 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.145090 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.145120 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.147537 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.148069 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.148072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.148074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.148205 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.148565 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.148568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.148570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.149143 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.149290 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.149674 (+ 384us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.149982 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":889,"spinlock_wait_cycles":20736,"thread_start_us":107,"threads_started":1}
I20260925 00:47:37.151824 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.135916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58726 (local address 127.6.72.124:37795)
0925 00:47:37.136368 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.136393 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.136634 (+ 241us) server_negotiation.cc:407] Connection header received
0925 00:47:37.137539 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.137562 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.138022 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.138384 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.140308 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.140803 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.141895 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.142207 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.142311 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.145564 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.145570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.145574 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.145601 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.147502 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.148028 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.148034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.148041 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.148205 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.148558 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.148561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.148563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.149143 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.149362 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.150107 (+ 745us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.150206 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1795,"server-negotiator.queue_time_us":289,"spinlock_wait_cycles":17920,"thread_start_us":139,"threads_started":1}
I20260925 00:47:37.154424 28747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.143617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58038 (local address 127.6.72.126:33683)
0925 00:47:37.143759 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.143763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.144010 (+ 247us) server_negotiation.cc:407] Connection header received
0925 00:47:37.144175 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.144180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.144227 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.144340 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.145022 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.145603 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.146888 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.147250 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.147370 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.150574 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.150578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.150581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.150613 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.152433 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.153016 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.153020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.153021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.153112 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.153462 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.153465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.153467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.153691 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.153795 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.154154 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.154272 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 00:47:37.154443 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.142925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:33683 (local address 127.0.0.1:58038)
0925 00:47:37.143893 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.143930 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:37.144062 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:37.144386 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:37.144391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:37.144791 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:37.144901 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:37.144907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.145778 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.145783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:37.146753 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:37.146767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.147384 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.147390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:37.147517 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.148021 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:37.148041 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:37.150436 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:37.152557 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:37.152564 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:37.152568 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:37.152895 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:37.153247 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:37.153251 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:37.153253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:37.153354 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:37.153813 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:37.153821 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:37.153947 (+ 126us) client_negotiation.cc:769] Sending connection context
0925 00:47:37.154090 (+ 143us) client_negotiation.cc:240] Negotiation successful
0925 00:47:37.154273 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":100,"threads_started":1}
I20260925 00:47:37.155730 28818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6: Bootstrap starting.
I20260925 00:47:37.156051 28895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:37.158427 28818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:37.159164 28818 log.cc:824] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6: Log is configured to *not* fsync() on all Append() calls
Sep 25 00:47:37 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:37.161520 28818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6: No bootstrap required, opened a new log
Sep 25 00:47:37 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297257, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
I20260925 00:47:37.164565 28901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.153701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58054 (local address 127.6.72.126:33683)
0925 00:47:37.154200 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.154212 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.154224 (+ 12us) server_negotiation.cc:407] Connection header received
0925 00:47:37.154271 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.154274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.154324 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.155359 (+ 1035us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.155470 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.156298 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.157095 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.157431 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.157554 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.159957 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.159962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.159966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.159993 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.161873 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.162335 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.162339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.162341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.162430 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.162755 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.162758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.162760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.162970 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.163163 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.164226 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.164381 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":108,"threads_started":1}
I20260925 00:47:37.165889 28818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } }
I20260925 00:47:37.166144 28818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae27fd311d0415093c6fef3fb3501d6, State: Initialized, Role: FOLLOWER
I20260925 00:47:37.166551 28818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } }
I20260925 00:47:37.167802 28903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } } }
I20260925 00:47:37.167965 28903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:37.168072 28818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:37.169064 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.159826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:40687 (local address 127.0.0.1:40698)
0925 00:47:37.159997 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.160010 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:37.160088 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:37.160376 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:37.160379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:37.160618 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:37.160757 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:37.160765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.161873 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.161877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:37.162595 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:37.162603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.163096 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.163100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:37.163200 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.163601 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:37.163612 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:37.165110 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:37.167413 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:37.167417 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:37.167419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:37.167660 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:37.167973 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:37.167976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:37.167978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:37.168067 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:37.168544 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:37.168547 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:37.168643 (+ 96us) client_negotiation.cc:769] Sending connection context
0925 00:47:37.168753 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 00:47:37.168892 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260925 00:47:37.169162 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.159907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40698 (local address 127.6.72.125:40687)
0925 00:47:37.160024 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.160031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.160048 (+ 17us) server_negotiation.cc:407] Connection header received
0925 00:47:37.160180 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.160184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.160264 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.160347 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.160909 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.161720 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.162713 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.162971 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.163081 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.165286 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.165290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.165293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.165320 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.167301 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.167772 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.167775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.167777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.167863 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.168153 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.168158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.168160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.168438 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.168541 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.168831 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.168969 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 00:47:37.170184 28895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 00:47:37 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
I20260925 00:47:37.172022 28750 master_runner.cc:186] Error getting master registration for 127.6.72.126:33683: OK, instance_id { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" instance_seqno: 1790297256657446 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
Sep 25 00:47:37 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297257, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
I20260925 00:47:37.175642 28750 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 00:47:37.176715 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.165923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.6.72.125:40687)
0925 00:47:37.166042 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.166046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.166065 (+ 19us) server_negotiation.cc:407] Connection header received
0925 00:47:37.166094 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.166096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.166157 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.166239 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.167123 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.167825 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.168935 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.169333 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.169413 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.172800 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.172803 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.172806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.172830 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.174533 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.175152 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.175156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.175157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.175245 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.175545 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.175551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.175556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.175845 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.176038 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.176403 (+ 365us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.176519 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260925 00:47:37.176539 28914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:37.176858 28914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:37.179314 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.170649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58736)
0925 00:47:37.170739 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.170754 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:37.170827 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:37.171141 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:37.171144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:37.171313 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:37.171375 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:37.171380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.172427 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.172432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:37.173144 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:37.173154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.173904 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.173910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:37.174000 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.174410 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:37.174428 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:37.175850 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:37.177869 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:37.177872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:37.177874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:37.178107 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:37.178393 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:37.178396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:37.178398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:37.178476 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:37.178868 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:37.178871 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:37.178956 (+ 85us) client_negotiation.cc:769] Sending connection context
0925 00:47:37.179059 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 00:47:37.179188 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 00:47:37.179466 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.170872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58736 (local address 127.6.72.124:37795)
0925 00:47:37.170967 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.170971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.170982 (+ 11us) server_negotiation.cc:407] Connection header received
0925 00:47:37.171013 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.171019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.171064 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.171131 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.171483 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.172343 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.173289 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.173673 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.173791 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.175969 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.175972 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.175974 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.175996 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.177764 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.178209 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.178214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.178223 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.178301 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.178566 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.178569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.178571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.178768 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.178868 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.179159 (+ 291us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.179302 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 00:47:37.180666 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:37.182663 28895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15: Bootstrap starting.
I20260925 00:47:37.181896 28717 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a9a90ccf65984e8ea42b1e899cb5a234"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58054
I20260925 00:47:37.185559 28895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:37.186494 28895 log.cc:824] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:37.189096 28895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15: No bootstrap required, opened a new log
I20260925 00:47:37.193356 28895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } }
I20260925 00:47:37.193650 28895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36dc5cb75c3b4f88b35d531e3b253f15, State: Initialized, Role: FOLLOWER
I20260925 00:47:37.194016 28895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } }
I20260925 00:47:37.194597 28917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } } }
I20260925 00:47:37.194753 28917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:37.195484 28895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 00:47:37.201493 28928 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:37.201583 28928 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:37.199836 28825 master_runner.cc:186] Error getting master registration for 127.6.72.126:33683: OK, instance_id { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" instance_seqno: 1790297256657446 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 00:47:37.203274 28825 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 00:47:37.295794 28743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } attempt: 4
Sep 25 00:47:37 dist-test-slave-w8nh krb5kdc[28662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297256, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
I20260925 00:47:37.306365 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.296978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.6.72.124:37795)
0925 00:47:37.297100 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.297105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.297123 (+ 18us) server_negotiation.cc:407] Connection header received
0925 00:47:37.297193 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.297196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.297242 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.297323 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.297891 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.298676 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.299490 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.299843 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.299937 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.302713 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.302720 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.302724 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.302754 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.304544 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.305033 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.305037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.305039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.305134 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.305448 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.305453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.305456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.305696 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.305791 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.306073 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.306190 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 00:47:37.306380 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.296485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:37795 (local address 127.0.0.1:58742)
0925 00:47:37.296769 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:37.296785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:37.296923 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:37.297358 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:37.297361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:37.297619 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:37.297754 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:37.297760 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.298777 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.298781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:37.299354 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:37.299362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.300083 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.300087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:37.300210 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.300865 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:37.300880 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:37.302514 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:37.304685 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:37.304688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:37.304690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:37.304951 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:37.305241 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:37.305246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:37.305248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:37.305345 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:37.305789 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:37.305792 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:37.305905 (+ 113us) client_negotiation.cc:769] Sending connection context
0925 00:47:37.306018 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 00:47:37.306191 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260925 00:47:37.310117 28743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234: Bootstrap starting.
I20260925 00:47:37.313154 28743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:37.314528 28743 log.cc:824] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:37.316953 28743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234: No bootstrap required, opened a new log
I20260925 00:47:37.320528 28743 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } }
I20260925 00:47:37.320785 28743 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a90ccf65984e8ea42b1e899cb5a234, State: Initialized, Role: FOLLOWER
I20260925 00:47:37.321152 28743 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } }
I20260925 00:47:37.321744 28931 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } } }
I20260925 00:47:37.322017 28931 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:37.322642 28743 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:37.322695 28717 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 00:47:37.322798 28717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:37.324272 28717 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } }
I20260925 00:47:37.325356 28717 leader_election.cc:288] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [CANDIDATE]: Term 1 election: Requested vote from peers 4ae27fd311d0415093c6fef3fb3501d6 (127.6.72.125:40687), 36dc5cb75c3b4f88b35d531e3b253f15 (127.6.72.124:37795)
I20260925 00:47:37.326020 28792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4ae27fd311d0415093c6fef3fb3501d6"
I20260925 00:47:37.326318 28792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:37.327013 28869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "36dc5cb75c3b4f88b35d531e3b253f15"
I20260925 00:47:37.327237 28869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:37.327766 28792 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a90ccf65984e8ea42b1e899cb5a234 in term 1.
I20260925 00:47:37.328581 28869 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a90ccf65984e8ea42b1e899cb5a234 in term 1.
I20260925 00:47:37.329339 28691 leader_election.cc:302] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36dc5cb75c3b4f88b35d531e3b253f15, a9a90ccf65984e8ea42b1e899cb5a234; no voters:
I20260925 00:47:37.330154 28931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:37.330438 28931 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [term 1 LEADER]: Becoming Leader. State: Replica: a9a90ccf65984e8ea42b1e899cb5a234, State: Running, Role: LEADER
I20260925 00:47:37.330845 28931 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } }
I20260925 00:47:37.332875 28931 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a90ccf65984e8ea42b1e899cb5a234. Latest consensus state: current_term: 1 leader_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } } }
I20260925 00:47:37.333070 28931 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:37.335278 28675 master_runner.cc:263] No leader master found from master a9a90ccf65984e8ea42b1e899cb5a234
I20260925 00:47:37.335350 28675 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 00:47:37.336854 28942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:37.338589 28942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:37.338749 28949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:37.339843 28949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:37.344317 28901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.333609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58070 (local address 127.6.72.126:33683)
0925 00:47:37.333879 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.333883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.334838 (+ 955us) server_negotiation.cc:407] Connection header received
0925 00:47:37.335039 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.335042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.335090 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.335168 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.336301 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.337074 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.338214 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.338538 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.338621 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.340619 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.340623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.340626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.340653 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.342390 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.342890 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.342894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.342896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.342979 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.343298 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.343302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.343303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.343526 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.343626 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.344004 (+ 378us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.344142 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260925 00:47:37.354707 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.334221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58750 (local address 127.6.72.124:37795)
0925 00:47:37.334332 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.334336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.345565 (+ 11229us) server_negotiation.cc:407] Connection header received
0925 00:47:37.345761 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.345765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.345828 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.345882 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.346762 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.347571 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.348700 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.349003 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.349082 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.350837 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.350841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.350844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.350873 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.352685 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.353207 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.353211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.353213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.353300 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.353608 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.353612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.353614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.353881 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.353992 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.354378 (+ 386us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.354531 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 00:47:37.357827 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:37.336587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.6.72.125:40687)
0925 00:47:37.336694 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.336697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.348566 (+ 11869us) server_negotiation.cc:407] Connection header received
0925 00:47:37.348799 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.348803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.348852 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.348907 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.349862 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.350635 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.351778 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:37.352114 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:37.352201 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:37.353960 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:37.353964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:37.353966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:37.353996 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:37.355795 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.356347 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.356351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.356352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.356450 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:37.356761 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:37.356764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:37.356766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:37.357016 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:37.357156 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:37.357537 (+ 381us) server_negotiation.cc:299] Negotiation successful
0925 00:47:37.357671 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 00:47:37.358183 28869 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [term 1 FOLLOWER]: Refusing update from remote peer a9a90ccf65984e8ea42b1e899cb5a234: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:37.358978 28947 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:37.359503 28792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [term 1 FOLLOWER]: Refusing update from remote peer a9a90ccf65984e8ea42b1e899cb5a234: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:37.360347 28947 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:37.363288 28917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a90ccf65984e8ea42b1e899cb5a234. Latest consensus state: current_term: 1 leader_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } } }
I20260925 00:47:37.363437 28917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:37.364557 28903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a90ccf65984e8ea42b1e899cb5a234. Latest consensus state: current_term: 1 leader_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } } }
I20260925 00:47:37.364790 28903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:37.366918 28947 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } } }
I20260925 00:47:37.367065 28947 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:37.368120 28903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } } }
I20260925 00:47:37.368373 28903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:37.368816 28954 mvcc.cc:205] Tried to move back new op lower bound from 7333057566131290112 to 7333057566031335424. Current Snapshot: MvccSnapshot[applied={T|T < 7333057566131290112}]
W20260925 00:47:37.371961 28771 tablet.cc:2369] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 00:47:37.375017 28848 tablet.cc:2369] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 00:47:37.376507 28949 catalog_manager.cc:1347] Generated new cluster ID: d1a4b83f746740e09b66e21c3eb1baf4
I20260925 00:47:37.376588 28949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:37.389107 28931 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } } }
I20260925 00:47:37.389279 28931 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:37.391402 28917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90ccf65984e8ea42b1e899cb5a234" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 33683 } } peers { permanent_uuid: "4ae27fd311d0415093c6fef3fb3501d6" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 40687 } } peers { permanent_uuid: "36dc5cb75c3b4f88b35d531e3b253f15" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 37795 } } }
I20260925 00:47:37.391538 28917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:37.392714 28955 mvcc.cc:205] Tried to move back new op lower bound from 7333057566131290112 to 7333057566031335424. Current Snapshot: MvccSnapshot[applied={T|T < 7333057566131290112}]
I20260925 00:47:37.393031 28957 mvcc.cc:205] Tried to move back new op lower bound from 7333057566131290112 to 7333057566031335424. Current Snapshot: MvccSnapshot[applied={T|T < 7333057566131290112 or (T in {7333057566131290112})}]
I20260925 00:47:37.414217 28949 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:37.415489 28949 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:37.424367 28949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9a90ccf65984e8ea42b1e899cb5a234: Generated new TSK 0
I20260925 00:47:37.425271 28949 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 00:47:37.453747 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.452053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40730 (local address 127.6.72.125:40687)
0925 00:47:37.452179 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.452183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.452795 (+ 612us) server_negotiation.cc:407] Connection header received
0925 00:47:37.452982 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.452986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.453034 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.453084 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.453589 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40730: BlockingRecv error: recv got EOF from 127.0.0.1:40730 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260925 00:47:37.453759 28901 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.452031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.6.72.126:33683)
0925 00:47:37.452180 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.452186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.452486 (+ 300us) server_negotiation.cc:407] Connection header received
0925 00:47:37.452745 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.452750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.452810 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.452873 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.453590 (+ 717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58080: BlockingRecv error: recv got EOF from 127.0.0.1:58080 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260925 00:47:37.454260 28898 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.452821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58764 (local address 127.6.72.124:37795)
0925 00:47:37.452949 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.452954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.453205 (+ 251us) server_negotiation.cc:407] Connection header received
0925 00:47:37.453399 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.453405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.453465 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.453524 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.454131 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58764: BlockingRecv error: recv got EOF from 127.0.0.1:58764 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 00:47:37.457933 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.456737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40746 (local address 127.6.72.125:40687)
0925 00:47:37.456851 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.456855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.457084 (+ 229us) server_negotiation.cc:407] Connection header received
0925 00:47:37.457213 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.457215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.457280 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.457353 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.457824 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40746: BlockingRecv error: recv got EOF from 127.0.0.1:40746 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260925 00:47:37.461951 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.461079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.6.72.125:40687)
0925 00:47:37.461202 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.461205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.461216 (+ 11us) server_negotiation.cc:407] Connection header received
0925 00:47:37.461258 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.461260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.461299 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.461389 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.461823 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40756: BlockingRecv error: recv got EOF from 127.0.0.1:40756 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260925 00:47:37.466872 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.465746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.6.72.125:40687)
0925 00:47:37.465859 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.465862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.466045 (+ 183us) server_negotiation.cc:407] Connection header received
0925 00:47:37.466170 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.466172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.466213 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.466281 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.466779 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40764: BlockingRecv error: recv got EOF from 127.0.0.1:40764 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260925 00:47:37.474049 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.473012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40768 (local address 127.6.72.125:40687)
0925 00:47:37.473173 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.473178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.473291 (+ 113us) server_negotiation.cc:407] Connection header received
0925 00:47:37.473382 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.473385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.473440 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.473533 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.473940 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40768: BlockingRecv error: recv got EOF from 127.0.0.1:40768 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260925 00:47:37.485435 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.484166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40776 (local address 127.6.72.125:40687)
0925 00:47:37.484343 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.484347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.484504 (+ 157us) server_negotiation.cc:407] Connection header received
0925 00:47:37.484675 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.484679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.484735 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.484828 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.485289 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40776: BlockingRecv error: recv got EOF from 127.0.0.1:40776 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260925 00:47:37.505163 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.503834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40780 (local address 127.6.72.125:40687)
0925 00:47:37.503997 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.504002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.504192 (+ 190us) server_negotiation.cc:407] Connection header received
0925 00:47:37.504362 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.504366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.504416 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.504489 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.505033 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40780: BlockingRecv error: recv got EOF from 127.0.0.1:40780 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260925 00:47:37.540742 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.539445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.6.72.125:40687)
0925 00:47:37.539618 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.539623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.539780 (+ 157us) server_negotiation.cc:407] Connection header received
0925 00:47:37.539965 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.539973 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.540022 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.540113 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.540628 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40794: BlockingRecv error: recv got EOF from 127.0.0.1:40794 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260925 00:47:37.608389 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.607025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40804 (local address 127.6.72.125:40687)
0925 00:47:37.607208 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.607213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.607357 (+ 144us) server_negotiation.cc:407] Connection header received
0925 00:47:37.607545 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.607549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.607616 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.607712 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.608205 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40804: BlockingRecv error: recv got EOF from 127.0.0.1:40804 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260925 00:47:37.739807 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.738775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40814 (local address 127.6.72.125:40687)
0925 00:47:37.738914 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.738918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.739010 (+ 92us) server_negotiation.cc:407] Connection header received
0925 00:47:37.739132 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.739135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.739185 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.739276 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.739695 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40814: BlockingRecv error: recv got EOF from 127.0.0.1:40814 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260925 00:47:37.999307 28823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:37.998125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.6.72.125:40687)
0925 00:47:37.998281 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:37.998284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:37.998300 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:47:37.998523 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:37.998527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:37.998606 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:37.998710 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:37.999197 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40820: BlockingRecv error: recv got EOF from 127.0.0.1:40820 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 00:47:38.178442 28914 catalog_manager.cc:1259] Loaded cluster ID: d1a4b83f746740e09b66e21c3eb1baf4
I20260925 00:47:38.178529 28914 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6: loading cluster ID for follower catalog manager: success
I20260925 00:47:38.180208 28914 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6: acquiring CA information for follower catalog manager: success
I20260925 00:47:38.181459 28914 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4ae27fd311d0415093c6fef3fb3501d6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 00:47:38.202895 28928 catalog_manager.cc:1259] Loaded cluster ID: d1a4b83f746740e09b66e21c3eb1baf4
I20260925 00:47:38.203018 28928 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15: loading cluster ID for follower catalog manager: success
I20260925 00:47:38.204722 28928 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15: acquiring CA information for follower catalog manager: success
I20260925 00:47:38.205925 28928 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36dc5cb75c3b4f88b35d531e3b253f15: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 00:47:38.517657 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:38.513568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46562 (local address 127.6.72.125:40687)
0925 00:47:38.513869 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:38.513876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:38.513895 (+ 19us) server_negotiation.cc:407] Connection header received
0925 00:47:38.513963 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:38.513966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:38.514091 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:38.514217 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:38.514507 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:38.515045 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:38.515950 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:38.516184 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:38.516358 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:38.516708 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:38.517177 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:38.517325 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:38.517387 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 00:47:38.517451 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":67,"threads_started":1}
I20260925 00:47:38.525233 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:38.521013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49988 (local address 127.6.72.124:37795)
0925 00:47:38.521264 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:38.521269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:38.521284 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:47:38.521481 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:38.521484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:38.521603 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:38.521735 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:38.521967 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:38.522496 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:38.523550 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:38.523790 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:38.523933 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:38.524349 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:38.524800 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:38.524958 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:38.524999 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 00:47:38.525059 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":66,"threads_started":1}
I20260925 00:47:38.532785 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:38.528433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.6.72.125:40687)
0925 00:47:38.528573 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:38.528599 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:38.528993 (+ 394us) server_negotiation.cc:407] Connection header received
0925 00:47:38.529120 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:38.529123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:38.529183 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:38.529272 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:38.529814 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:38.530610 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:38.531537 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:38.531759 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:38.531880 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:38.532158 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:38.532296 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:38.532389 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:38.532559 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 00:47:38.532633 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 00:47:38.533677 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:38.528819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50000 (local address 127.6.72.124:37795)
0925 00:47:38.528950 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:38.528954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:38.529241 (+ 287us) server_negotiation.cc:407] Connection header received
0925 00:47:38.529322 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:38.529325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:38.529381 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:38.529451 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:38.530744 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:38.531240 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:38.532462 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:38.532804 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:38.532901 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:38.533194 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:38.533277 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:38.533355 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:38.533491 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 00:47:38.533551 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 00:47:38.534329 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:38.528395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.6.72.126:33683)
0925 00:47:38.528660 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:38.528665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:38.528915 (+ 250us) server_negotiation.cc:407] Connection header received
0925 00:47:38.529110 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:38.529114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:38.529303 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:38.529426 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:38.529747 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:38.530620 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:38.532636 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:38.533013 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:38.533109 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:38.533522 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:38.533923 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:38.534034 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:38.534112 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 00:47:38.534171 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":57,"threads_started":1}
I20260925 00:47:38.537407 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 28675
I20260925 00:47:38.544162 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 28750
I20260925 00:47:38.550251 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 28825
2026-09-25T00:47:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3268 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 00:47:38 dist-test-slave-w8nh krb5kdc[29044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 00:47:38 dist-test-slave-w8nh krb5kdc[29044](info): set up 2 sockets
Sep 25 00:47:38 dist-test-slave-w8nh krb5kdc[29044](info): commencing operation
krb5kdc: starting...
W20260925 00:47:39.596702 6433 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/test-user.keytab.
Sep 25 00:47:39 dist-test-slave-w8nh krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T00:47:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T00:47:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.72.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
Entry for principal kudu/127.6.72.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab.
I20260925 00:47:39.724129 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:39591
--webserver_interface=127.6.72.126
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:39591,127.6.72.125:33767,127.6.72.124:42389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 00:47:39.833505 29057 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 00:47:39.833796 29057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:39.833822 29057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:39.833860 29057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:39.837759 29057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:39.837805 29057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:39.837823 29057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:39.837841 29057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:39.837855 29057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:39.843058 29057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:39591,127.6.72.125:33767,127.6.72.124:42389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.126.keytab
--principal=kudu/127.6.72.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.126:39591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.72.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:39.844033 29057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:39.845002 29057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:39.853353 29062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:39.853421 29065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:39.853335 29063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:39.853530 29057 server_base.cc:1034] running on GCE node
I20260925 00:47:39.854159 29057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:39.855214 29057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:39.856393 29057 hybrid_clock.cc:648] HybridClock initialized: now 1790297259856366 us; error 38 us; skew 500 ppm
Sep 25 00:47:39 dist-test-slave-w8nh krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297259, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:39.859120 29057 init.cc:377] Logged in from keytab as kudu/127.6.72.126@KRBTEST.COM (short username kudu)
I20260925 00:47:39.860033 29057 webserver.cc:460] Webserver started at http://127.6.72.126:37105/ using document root <none> and password file <none>
I20260925 00:47:39.860659 29057 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:39.860757 29057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:39.860994 29057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:39.862994 29057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/data/instance:
uuid: "a68efade82b64cf3bda17b5a8ff5af81"
format_stamp: "Formatted at 2026-09-25 00:47:39 on dist-test-slave-w8nh"
server_key: "40eafcfc3f77ff2679fd127d3997391a"
server_key_iv: "aaaa215dac4cde509de61f9f4f1a4863"
server_key_version: "encryptionkey@0"
I20260925 00:47:39.863567 29057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/wal/instance:
uuid: "a68efade82b64cf3bda17b5a8ff5af81"
format_stamp: "Formatted at 2026-09-25 00:47:39 on dist-test-slave-w8nh"
server_key: "40eafcfc3f77ff2679fd127d3997391a"
server_key_iv: "aaaa215dac4cde509de61f9f4f1a4863"
server_key_version: "encryptionkey@0"
I20260925 00:47:39.867383 29057 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 00:47:39.869889 29072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:39.871084 29057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 00:47:39.871220 29057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/wal
uuid: "a68efade82b64cf3bda17b5a8ff5af81"
format_stamp: "Formatted at 2026-09-25 00:47:39 on dist-test-slave-w8nh"
server_key: "40eafcfc3f77ff2679fd127d3997391a"
server_key_iv: "aaaa215dac4cde509de61f9f4f1a4863"
server_key_version: "encryptionkey@0"
I20260925 00:47:39.871341 29057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:39.881418 29057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:39.884897 29057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:39.885114 29057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:39.892212 29057 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.126:39591
I20260925 00:47:39.892572 29124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.126:39591 every 8 connection(s)
I20260925 00:47:39.893242 29057 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/data/info.pb
I20260925 00:47:39.896008 29125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:39.899705 29125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:39.900502 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 29057
I20260925 00:47:39.900619 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-0/wal/instance
I20260925 00:47:39.900943 6433 external_mini_cluster.cc:1428] Setting key 6ac0d6d6155dd50c53d7385713bd1330
WARNING: no policy specified for kudu/127.6.72.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.125@KRBTEST.COM" created.
Sep 25 00:47:39 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297259, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:39.918439 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:39.902355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51056 (local address 127.6.72.126:39591)
0925 00:47:39.902853 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:39.902875 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:39.903146 (+ 271us) server_negotiation.cc:407] Connection header received
0925 00:47:39.904434 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:39.904439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:39.904625 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:39.904793 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:39.905930 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:39.906804 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:39.908352 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:39.908704 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:39.908827 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:39.913466 (+ 4639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:39.913471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:39.913482 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:39.913504 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:39.915400 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:39.915881 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:39.915891 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:39.915895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:39.916034 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:39.916370 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:39.916376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:39.916379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:39.916932 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:39.917078 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:39.917460 (+ 382us) server_negotiation.cc:299] Negotiation successful
0925 00:47:39.917636 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":140,"threads_started":1}
I20260925 00:47:39.918437 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:39.902355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:39591 (local address 127.0.0.1:51056)
0925 00:47:39.902979 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:39.903028 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:39.904128 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:39.904849 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:39.904858 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:39.905372 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:39.905751 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:39.905769 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:39.906941 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:39.906946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:39.908178 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:39.908191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:39.908879 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:39.908885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:39.909045 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:39.909709 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:39.909730 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:39.913308 (+ 3578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:39.915505 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:39.915513 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:39.915520 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:39.915734 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:39.916128 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:39.916130 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:39.916131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:39.916231 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:39.917051 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:39.917056 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:39.917176 (+ 120us) client_negotiation.cc:769] Sending connection context
0925 00:47:39.917314 (+ 138us) client_negotiation.cc:240] Negotiation successful
0925 00:47:39.917563 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"spinlock_wait_cycles":31488,"thread_start_us":154,"threads_started":1}
I20260925 00:47:39.921092 29125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:39.921852 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:39.921570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:33767 (local address 127.0.0.1:47172)
0925 00:47:39.921705 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:39.921792 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:33767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260925 00:47:39.922108 29075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.72.125:33767: connect: Connection refused (error 111)
I20260925 00:47:39.922917 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:39.922711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:33767 (local address 127.0.0.1:47184)
0925 00:47:39.922822 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:39.922864 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:33767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 00:47:39.923134 29125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.125:33767: Network error: Client connection negotiation failed: client connection to 127.6.72.125:33767: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab.
Entry for principal kudu/127.6.72.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab.
I20260925 00:47:39.930774 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.125:33767
--webserver_interface=127.6.72.125
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab
--principal=kudu/127.6.72.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:39591,127.6.72.125:33767,127.6.72.124:42389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:47:39.972390 29125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } attempt: 1
I20260925 00:47:39.973336 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:39.972993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:33767 (local address 127.0.0.1:47186)
0925 00:47:39.973195 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:39.973259 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:33767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260925 00:47:39.974052 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:39.973842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:33767 (local address 127.0.0.1:47200)
0925 00:47:39.973948 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:39.973999 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:33767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 00:47:39.974200 29125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.125:33767: Network error: Client connection negotiation failed: client connection to 127.6.72.125:33767: connect: Connection refused (error 111)
I20260925 00:47:40.042373 29125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } attempt: 2
W20260925 00:47:40.042649 29132 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 00:47:40.042990 29132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:40.043056 29132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:40.043109 29132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 00:47:40.043313 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.042938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:33767 (local address 127.0.0.1:47212)
0925 00:47:40.043140 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.043214 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:33767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260925 00:47:40.044050 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.043835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:33767 (local address 127.0.0.1:47228)
0925 00:47:40.043937 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.043985 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.125:33767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 00:47:40.044292 29125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.125:33767: Network error: Client connection negotiation failed: client connection to 127.6.72.125:33767: connect: Connection refused (error 111)
W20260925 00:47:40.047695 29132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:40.047755 29132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:40.047775 29132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:40.047825 29132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:40.047857 29132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:40.053390 29132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:39591,127.6.72.125:33767,127.6.72.124:42389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.125.keytab
--principal=kudu/127.6.72.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.125:33767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.72.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:40.054476 29132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:40.055507 29132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:40.064008 29138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:40.063993 29137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:40.063997 29140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:40.064222 29132 server_base.cc:1034] running on GCE node
I20260925 00:47:40.064879 29132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:40.066015 29132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:40.067202 29132 hybrid_clock.cc:648] HybridClock initialized: now 1790297260067172 us; error 44 us; skew 500 ppm
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297260, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:40.070016 29132 init.cc:377] Logged in from keytab as kudu/127.6.72.125@KRBTEST.COM (short username kudu)
I20260925 00:47:40.070950 29132 webserver.cc:460] Webserver started at http://127.6.72.125:37679/ using document root <none> and password file <none>
I20260925 00:47:40.071552 29132 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:40.071645 29132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:40.071880 29132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:40.073975 29132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/data/instance:
uuid: "1c6292626f374eb4af128e0e78922fef"
format_stamp: "Formatted at 2026-09-25 00:47:40 on dist-test-slave-w8nh"
server_key: "8c31c6d774ffcf72d4fe73f38ad2960f"
server_key_iv: "216d43ab44dd5649386ada1b9341caf1"
server_key_version: "encryptionkey@0"
I20260925 00:47:40.074558 29132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/wal/instance:
uuid: "1c6292626f374eb4af128e0e78922fef"
format_stamp: "Formatted at 2026-09-25 00:47:40 on dist-test-slave-w8nh"
server_key: "8c31c6d774ffcf72d4fe73f38ad2960f"
server_key_iv: "216d43ab44dd5649386ada1b9341caf1"
server_key_version: "encryptionkey@0"
I20260925 00:47:40.078367 29132 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 00:47:40.081633 29147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:40.083388 29132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 00:47:40.083530 29132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/wal
uuid: "1c6292626f374eb4af128e0e78922fef"
format_stamp: "Formatted at 2026-09-25 00:47:40 on dist-test-slave-w8nh"
server_key: "8c31c6d774ffcf72d4fe73f38ad2960f"
server_key_iv: "216d43ab44dd5649386ada1b9341caf1"
server_key_version: "encryptionkey@0"
I20260925 00:47:40.083665 29132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:40.105700 29132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:40.109047 29132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:40.109215 29132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:40.116287 29132 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.125:33767
I20260925 00:47:40.116580 29199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.125:33767 every 8 connection(s)
I20260925 00:47:40.117334 29132 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/data/info.pb
I20260925 00:47:40.120103 29200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:40.123827 29200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:40.127467 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 29132
I20260925 00:47:40.127593 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-1/wal/instance
I20260925 00:47:40.127918 6433 external_mini_cluster.cc:1428] Setting key a61becfd5ed5e558fed459d9a0f8bc25
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297260, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:40.135514 29125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } attempt: 3
WARNING: no policy specified for kudu/127.6.72.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.72.124@KRBTEST.COM" created.
I20260925 00:47:40.140174 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.125809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.6.72.126:39591)
0925 00:47:40.126005 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.126010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.126939 (+ 929us) server_negotiation.cc:407] Connection header received
0925 00:47:40.128100 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.128103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.128153 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.128227 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.129917 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.130780 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.131913 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.132368 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.132483 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.134961 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.134964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.134967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.135000 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.137753 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.138366 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.138369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.138370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.138441 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.138822 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.138828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.138833 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.139084 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.139180 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.139826 (+ 646us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.139962 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":71}
I20260925 00:47:40.141460 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.126271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:39591 (local address 127.0.0.1:51060)
0925 00:47:40.126792 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.126835 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:40.127870 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:40.128473 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:40.128484 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:40.129017 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:40.129712 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.129737 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.130932 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.130934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.131781 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.131790 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.132516 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.132519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.132656 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.133340 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:40.133356 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:40.134778 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:40.137895 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.137902 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.137925 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.138179 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.138537 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.138543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.138546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.138685 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.139178 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:40.139185 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:40.139513 (+ 328us) client_negotiation.cc:769] Sending connection context
0925 00:47:40.139729 (+ 216us) client_negotiation.cc:240] Negotiation successful
0925 00:47:40.140107 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":116,"threads_started":1}
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297259, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
I20260925 00:47:40.143304 29200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:40.147482 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.136053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:33767 (local address 127.0.0.1:47230)
0925 00:47:40.136206 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.136230 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:40.136362 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:40.137496 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:40.137501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:40.137750 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:40.137862 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.137869 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.138897 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.138903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.139645 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.139655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.140672 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.140677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.140827 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.141325 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:40.141339 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:40.142963 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:40.145639 (+ 2676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.145644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.145648 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.145933 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.146269 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.146273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.146276 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.146379 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.146880 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:40.146883 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:40.147002 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 00:47:40.147126 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 00:47:40.147279 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":3}
I20260925 00:47:40.147564 29204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.136546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.6.72.125:33767)
0925 00:47:40.136969 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.136982 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.137008 (+ 26us) server_negotiation.cc:407] Connection header received
0925 00:47:40.137059 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.137066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.137293 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.137491 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.138004 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.138754 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.139818 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.140509 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.140672 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.143143 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.143151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.143164 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.143191 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.145519 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.146054 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.146061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.146063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.146181 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.146491 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.146494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.146496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.146753 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.146902 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.147221 (+ 319us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.147366 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":167,"threads_started":1}
I20260925 00:47:40.149955 29125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:40.150763 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.150475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51184)
0925 00:47:40.150592 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.150644 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297260, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
I20260925 00:47:40.151546 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.151291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51196)
0925 00:47:40.151437 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.151484 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260925 00:47:40.151717 29125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:42389: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
I20260925 00:47:40.154956 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.144060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:33767 (local address 127.0.0.1:47234)
0925 00:47:40.144219 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.144235 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:40.144343 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:40.144937 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:40.144943 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:40.145141 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:40.145289 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.145299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.146231 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.146237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.147118 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.147129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.148271 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.148276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.148412 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.148832 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:40.148852 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:40.151336 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:40.153297 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.153304 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.153308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.153514 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.153811 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.153817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.153819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.153908 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.154323 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:40.154326 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:40.154419 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 00:47:40.154543 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 00:47:40.154752 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":41}
I20260925 00:47:40.154994 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.144213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.6.72.125:33767)
0925 00:47:40.144594 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.144656 (+ 62us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.144673 (+ 17us) server_negotiation.cc:407] Connection header received
0925 00:47:40.144713 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.144716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.144784 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.144893 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.145430 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.146106 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.147255 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.148113 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.148262 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.151500 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.151505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.151507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.151539 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.153184 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.153616 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.153620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.153622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.153713 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.154001 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.154007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.154010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.154218 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.154338 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.154590 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.154821 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":297,"thread_start_us":118,"threads_started":1}
I20260925 00:47:40.155910 29200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:40.156641 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.156389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51208)
0925 00:47:40.156497 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.156583 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.72.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab.
Entry for principal kudu/127.6.72.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab.
W20260925 00:47:40.156876 29150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
I20260925 00:47:40.157681 6433 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklhk22w/build/debug/bin/kudu
/tmp/dist-test-tasklhk22w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.124:42389
--webserver_interface=127.6.72.124
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab
--principal=kudu/127.6.72.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.72.84:37103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.72.126:39591,127.6.72.125:33767,127.6.72.124:42389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 00:47:40.157804 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.157584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51224)
0925 00:47:40.157691 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.157736 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 00:47:40.157970 29200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:42389: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
I20260925 00:47:40.182955 29125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } attempt: 1
I20260925 00:47:40.183872 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.183577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51238)
0925 00:47:40.183735 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.183796 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 00:47:40.184593 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.184404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51244)
0925 00:47:40.184500 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.184543 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 00:47:40.184736 29125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:42389: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
I20260925 00:47:40.207278 29200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } attempt: 1
I20260925 00:47:40.208389 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.207916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51258)
0925 00:47:40.208155 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.208255 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260925 00:47:40.209173 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.208953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51264)
0925 00:47:40.209063 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.209114 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 00:47:40.209352 29200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:42389: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
I20260925 00:47:40.259918 29125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } attempt: 2
I20260925 00:47:40.260850 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.260571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51266)
0925 00:47:40.260715 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.260775 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 00:47:40.261550 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.261351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51278)
0925 00:47:40.261452 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.261501 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 00:47:40.261677 29125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:42389: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
W20260925 00:47:40.272706 29208 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 00:47:40.273072 29208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 00:47:40.273103 29208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 00:47:40.273144 29208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 00:47:40.277374 29208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 00:47:40.277437 29208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 00:47:40.277458 29208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 00:47:40.277475 29208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 00:47:40.277491 29208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 00:47:40.277556 29200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } attempt: 2
I20260925 00:47:40.278358 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.278049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51290)
0925 00:47:40.278201 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.278272 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 00:47:40.279105 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.278883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51298)
0925 00:47:40.278991 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.279039 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 00:47:40.279255 29200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.72.124:42389: Network error: Client connection negotiation failed: client connection to 127.6.72.124:42389: connect: Connection refused (error 111)
I20260925 00:47:40.283298 29208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.72.84:37103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.72.126:39591,127.6.72.125:33767,127.6.72.124:42389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/krb5kdc/kudu_127.6.72.124.keytab
--principal=kudu/127.6.72.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.72.124:42389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.72.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 25 Sep 2026 00:05:21 UTC on ccycloud.quasar-ertzuq.root.comops.site
build id 4671574
I20260925 00:47:40.284462 29208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 00:47:40.285497 29208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 00:47:40.294425 29216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 00:47:40.294427 29214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:40.294590 29208 server_base.cc:1034] running on GCE node
W20260925 00:47:40.294791 29213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 00:47:40.295403 29208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 00:47:40.296622 29208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 00:47:40.297847 29208 hybrid_clock.cc:648] HybridClock initialized: now 1790297260297834 us; error 32 us; skew 500 ppm
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297260, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 00:47:40.300725 29208 init.cc:377] Logged in from keytab as kudu/127.6.72.124@KRBTEST.COM (short username kudu)
I20260925 00:47:40.301730 29208 webserver.cc:460] Webserver started at http://127.6.72.124:46275/ using document root <none> and password file <none>
I20260925 00:47:40.302311 29208 fs_manager.cc:359] Metadata directory not provided
I20260925 00:47:40.302404 29208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 00:47:40.302636 29208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 00:47:40.304863 29208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/data/instance:
uuid: "63a80b356b4f4cceb799c0c7d3361a79"
format_stamp: "Formatted at 2026-09-25 00:47:40 on dist-test-slave-w8nh"
server_key: "453900eaec652223accfed3143a8e3e8"
server_key_iv: "53ea8c55280990a5eb2906328b4a9bd4"
server_key_version: "encryptionkey@0"
I20260925 00:47:40.305442 29208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/wal/instance:
uuid: "63a80b356b4f4cceb799c0c7d3361a79"
format_stamp: "Formatted at 2026-09-25 00:47:40 on dist-test-slave-w8nh"
server_key: "453900eaec652223accfed3143a8e3e8"
server_key_iv: "53ea8c55280990a5eb2906328b4a9bd4"
server_key_version: "encryptionkey@0"
I20260925 00:47:40.309171 29208 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 00:47:40.311545 29223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 00:47:40.312872 29208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260925 00:47:40.313010 29208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/data,/tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/wal
uuid: "63a80b356b4f4cceb799c0c7d3361a79"
format_stamp: "Formatted at 2026-09-25 00:47:40 on dist-test-slave-w8nh"
server_key: "453900eaec652223accfed3143a8e3e8"
server_key_iv: "53ea8c55280990a5eb2906328b4a9bd4"
server_key_version: "encryptionkey@0"
I20260925 00:47:40.313123 29208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 00:47:40.334296 29208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 00:47:40.337404 29208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 00:47:40.337626 29208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 00:47:40.344342 29208 rpc_server.cc:307] RPC server started. Bound to: 127.6.72.124:42389
I20260925 00:47:40.344652 29275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.72.124:42389 every 8 connection(s)
I20260925 00:47:40.345327 29208 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/data/info.pb
I20260925 00:47:40.348052 29276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 00:47:40.351744 29276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:40.354442 6433 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklhk22w/build/debug/bin/kudu as pid 29208
I20260925 00:47:40.354589 6433 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklhk22w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790296971355272-6433-0/minicluster-data/master-2/wal/instance
I20260925 00:47:40.354907 6433 external_mini_cluster.cc:1428] Setting key 6f132ac0c64f080986e5c71b6982c9c2
I20260925 00:47:40.360836 29125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } attempt: 3
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297260, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297259, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.126@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
I20260925 00:47:40.368855 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.353665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51072 (local address 127.6.72.126:39591)
0925 00:47:40.353819 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.353822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.354670 (+ 848us) server_negotiation.cc:407] Connection header received
0925 00:47:40.355944 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.355950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.356013 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.356096 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.357451 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.357940 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.359049 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.359341 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.359419 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.363353 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.363356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.363358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.363380 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.365516 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.367272 (+ 1756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.367277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.367279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.367372 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.367721 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.367725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.367728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.367936 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.367995 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.368525 (+ 530us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.368649 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 00:47:40.369643 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.354023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.126:39591 (local address 127.0.0.1:51072)
0925 00:47:40.354513 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.354568 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:40.355699 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:40.356273 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:40.356282 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:40.356747 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:40.357299 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.357312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.358049 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.358052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.358925 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.358933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.359479 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.359483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.359650 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.360777 (+ 1127us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:40.360793 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:40.363163 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:40.365638 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.365647 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.365660 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.365968 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.367469 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.367474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.367476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.367601 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.368063 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:40.368068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:40.368287 (+ 219us) client_negotiation.cc:769] Sending connection context
0925 00:47:40.368455 (+ 168us) client_negotiation.cc:240] Negotiation successful
0925 00:47:40.368877 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":156,"thread_start_us":160,"threads_started":1}
I20260925 00:47:40.369647 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.355562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.6.72.124:42389)
0925 00:47:40.355939 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.355951 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.355989 (+ 38us) server_negotiation.cc:407] Connection header received
0925 00:47:40.356222 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.356227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.356462 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.356641 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.357313 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.358333 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.359170 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.359600 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.359696 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.363230 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.363238 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.363253 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.363293 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.365628 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.366139 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.366145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.366147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.366323 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.366664 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.366668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.366670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.367287 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.367518 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.368388 (+ 870us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.368743 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":285,"spinlock_wait_cycles":2816,"thread_start_us":116,"threads_started":1}
I20260925 00:47:40.370415 29200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } attempt: 3
I20260925 00:47:40.371254 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.361259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51314)
0925 00:47:40.361464 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.361480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:40.361555 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:40.361999 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:40.362002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:40.362137 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:40.362255 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.362277 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.363258 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.363261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.363990 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.364002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.364609 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.364615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.364718 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.365140 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:40.365156 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:40.366516 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:40.369229 (+ 2713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.369233 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.369235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.369484 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.369971 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.369976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.369978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.370066 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.370715 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:40.370718 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:40.370844 (+ 126us) client_negotiation.cc:769] Sending connection context
0925 00:47:40.370965 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 00:47:40.371093 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260925 00:47:40.371822 29281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.361351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51314 (local address 127.6.72.124:42389)
0925 00:47:40.361749 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.361758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.361773 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:47:40.361823 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.361825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.361874 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.361962 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.362397 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.363164 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.364126 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.364484 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.364600 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.366672 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.366676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.366679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.366698 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.369111 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.369755 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.369769 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.369770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.369863 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.370156 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.370158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.370159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.370309 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.370715 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.371547 (+ 832us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.371662 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":132,"threads_started":1}
I20260925 00:47:40.372077 29276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:40.375602 29125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81: Bootstrap starting.
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297260, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297260, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.125@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
I20260925 00:47:40.378513 29125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:40.379446 29125 log.cc:824] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81: Log is configured to *not* fsync() on all Append() calls
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.126@KRBTEST.COM
I20260925 00:47:40.382686 29125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81: No bootstrap required, opened a new log
I20260925 00:47:40.382717 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.372632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47240 (local address 127.6.72.125:33767)
0925 00:47:40.372733 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.372736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.372976 (+ 240us) server_negotiation.cc:407] Connection header received
0925 00:47:40.373077 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.373081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.373121 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.373215 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.373593 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.374179 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.375366 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.375730 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.375814 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.378938 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.378944 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.378951 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.378982 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.380726 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.381213 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.381219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.381222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.381299 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.381595 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.381598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.381615 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.381940 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.382013 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.382394 (+ 381us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.382530 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 00:47:40.382898 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.370948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51320)
0925 00:47:40.371131 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.371143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:40.371230 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:40.371704 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:40.371707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:40.371892 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:40.372003 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.372010 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.375072 (+ 3062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.375075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.375830 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.375842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.376515 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.376519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.376593 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.377044 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:40.377062 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:40.378784 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:40.380734 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.380737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.380739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.380981 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.381363 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.381369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.381375 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.381456 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.382459 (+ 1003us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:40.382462 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:40.382550 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 00:47:40.382639 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 00:47:40.382766 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260925 00:47:40.383195 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.372538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.125:33767 (local address 127.0.0.1:47240)
0925 00:47:40.372891 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.372903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:40.373004 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:40.373226 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:40.373229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:40.373397 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:40.373479 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.373484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.374319 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.374322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.375238 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.375247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.375958 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.375961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.376074 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.376592 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:40.376608 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:40.378765 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:40.380849 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.380852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.380854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.381091 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.381412 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.381416 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.381418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.381492 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.382101 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:40.382103 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:40.382199 (+ 96us) client_negotiation.cc:769] Sending connection context
0925 00:47:40.382851 (+ 652us) client_negotiation.cc:240] Negotiation successful
0925 00:47:40.382985 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280}
I20260925 00:47:40.384426 29276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } has no permanent_uuid. Determining permanent_uuid...
I20260925 00:47:40.384845 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.370960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.6.72.124:42389)
0925 00:47:40.371489 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.371492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.371505 (+ 13us) server_negotiation.cc:407] Connection header received
0925 00:47:40.371534 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.371537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.371574 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.372115 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.372197 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.374870 (+ 2673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.375984 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.376387 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.376797 (+ 410us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.378936 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.378940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.378942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.378965 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.380623 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.381168 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.381172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.381175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.381256 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.381879 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.381883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.381885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.382110 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.384044 (+ 1934us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.384114 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.384263 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463}
I20260925 00:47:40.385320 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.372911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51084 (local address 127.6.72.126:39591)
0925 00:47:40.373014 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.373018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.373080 (+ 62us) server_negotiation.cc:407] Connection header received
0925 00:47:40.373253 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.373256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.373313 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.373361 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.374267 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.375095 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.376618 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.377247 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.377328 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.381338 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.381341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.381344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.381366 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.383036 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.383517 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.383524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.383529 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.383625 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.383932 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.383935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.383937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.384153 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.384359 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.385034 (+ 675us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.385155 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 00:47:40.387517 29200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef: Bootstrap starting.
I20260925 00:47:40.387611 29125 raft_consensus.cc:348] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } }
I20260925 00:47:40.387838 29125 raft_consensus.cc:729] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a68efade82b64cf3bda17b5a8ff5af81, State: Initialized, Role: FOLLOWER
I20260925 00:47:40.388177 29125 consensus_queue.cc:260] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } }
I20260925 00:47:40.389652 29125 sys_catalog.cc:564] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:40.390285 29200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef: Neither blocks nor log segments found. Creating new log.
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297260, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.72.124@KRBTEST.COM for kudu/127.6.72.124@KRBTEST.COM
I20260925 00:47:40.391131 29200 log.cc:824] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:40.393836 29200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef: No bootstrap required, opened a new log
I20260925 00:47:40.394817 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.384970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.72.124:42389 (local address 127.0.0.1:51332)
0925 00:47:40.385685 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0925 00:47:40.385699 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 00:47:40.385786 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 00:47:40.386055 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 00:47:40.386057 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 00:47:40.386220 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 00:47:40.386302 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.386309 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.387364 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.387367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.388224 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 00:47:40.388234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.389017 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.389020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 00:47:40.389088 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.389548 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 00:47:40.389562 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 00:47:40.391186 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 00:47:40.393204 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.393207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.393209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.393426 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.393708 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 00:47:40.393711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 00:47:40.393712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 00:47:40.393795 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 00:47:40.394320 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 00:47:40.394323 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 00:47:40.394406 (+ 83us) client_negotiation.cc:769] Sending connection context
0925 00:47:40.394504 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 00:47:40.394622 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629}
I20260925 00:47:40.395285 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.385185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.6.72.124:42389)
0925 00:47:40.385595 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.385599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.385777 (+ 178us) server_negotiation.cc:407] Connection header received
0925 00:47:40.385883 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.385885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.385919 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.385973 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.386416 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.387252 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.388490 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.388833 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.388926 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.391370 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.391374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.391378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.391403 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.393096 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.393528 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.393530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.393532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.393606 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.393928 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.393932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.393933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.394141 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.394258 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.395046 (+ 788us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.395147 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349}
I20260925 00:47:40.396358 29283 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } } }
I20260925 00:47:40.396497 29283 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [sys.catalog]: This master's current role is: FOLLOWER
Sep 25 00:47:40 dist-test-slave-w8nh krb5kdc[29044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790297259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.72.125@KRBTEST.COM
I20260925 00:47:40.398502 29276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79: Bootstrap starting.
I20260925 00:47:40.394941 29057 master_runner.cc:186] Error getting master registration for 127.6.72.125:33767: OK, instance_id { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" instance_seqno: 1790297260114191 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 00:47:40.398993 29057 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 00:47:40.400635 29295 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:40.400703 29295 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:40.401432 29276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79: Neither blocks nor log segments found. Creating new log.
I20260925 00:47:40.402335 29276 log.cc:824] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79: Log is configured to *not* fsync() on all Append() calls
I20260925 00:47:40.402411 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.389010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.6.72.125:33767)
0925 00:47:40.389159 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.389163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.391743 (+ 2580us) server_negotiation.cc:407] Connection header received
0925 00:47:40.392003 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.392006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.392059 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.392114 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.393037 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.393727 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.394808 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.395114 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.395203 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.398184 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.398188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.398190 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.398214 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.399970 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.400557 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.400560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.400562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.400650 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.400988 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.400991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.400993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.401228 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.401441 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.402120 (+ 679us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.402256 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 00:47:40.404757 29276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79: No bootstrap required, opened a new log
I20260925 00:47:40.405565 6433 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 00:47:40.406740 29099 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a68efade82b64cf3bda17b5a8ff5af81"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51084
I20260925 00:47:40.406991 29099 raft_consensus.cc:482] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 00:47:40.407115 29099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:40.407152 29200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } }
I20260925 00:47:40.407404 29200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c6292626f374eb4af128e0e78922fef, State: Initialized, Role: FOLLOWER
I20260925 00:47:40.407764 29200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } }
I20260925 00:47:40.408313 29298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } } }
I20260925 00:47:40.408504 29298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:40.408545 29099 raft_consensus.cc:504] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } }
I20260925 00:47:40.408792 29276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } }
I20260925 00:47:40.408998 29276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a80b356b4f4cceb799c0c7d3361a79, State: Initialized, Role: FOLLOWER
I20260925 00:47:40.409245 29200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:40.409319 29276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } }
I20260925 00:47:40.409464 29099 leader_election.cc:288] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [CANDIDATE]: Term 1 election: Requested vote from peers 1c6292626f374eb4af128e0e78922fef (127.6.72.125:33767), 63a80b356b4f4cceb799c0c7d3361a79 (127.6.72.124:42389)
I20260925 00:47:40.410706 29276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 00:47:40.410804 29174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a68efade82b64cf3bda17b5a8ff5af81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1c6292626f374eb4af128e0e78922fef"
I20260925 00:47:40.411029 29174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:40.412336 29174 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a68efade82b64cf3bda17b5a8ff5af81 in term 1.
I20260925 00:47:40.413002 29250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a68efade82b64cf3bda17b5a8ff5af81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "63a80b356b4f4cceb799c0c7d3361a79"
I20260925 00:47:40.413741 29250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [term 0 FOLLOWER]: Advancing to term 1
I20260925 00:47:40.414013 29075 leader_election.cc:302] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c6292626f374eb4af128e0e78922fef, a68efade82b64cf3bda17b5a8ff5af81; no voters:
I20260925 00:47:40.415114 29250 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a68efade82b64cf3bda17b5a8ff5af81 in term 1.
I20260925 00:47:40.415545 29283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 00:47:40.415761 29283 raft_consensus.cc:686] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [term 1 LEADER]: Becoming Leader. State: Replica: a68efade82b64cf3bda17b5a8ff5af81, State: Running, Role: LEADER
I20260925 00:47:40.416085 29283 consensus_queue.cc:237] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } }
W20260925 00:47:40.416366 29325 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:40.416442 29325 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:40.416638 29299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } } }
I20260925 00:47:40.416774 29299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:40.420801 29208 master_runner.cc:263] No leader master found from master 63a80b356b4f4cceb799c0c7d3361a79
I20260925 00:47:40.420866 29208 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 00:47:40.421821 29283 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a68efade82b64cf3bda17b5a8ff5af81. Latest consensus state: current_term: 1 leader_uuid: "a68efade82b64cf3bda17b5a8ff5af81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } } }
I20260925 00:47:40.421937 29283 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:40.426095 29332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 00:47:40.427374 29332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 00:47:40.432894 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.412431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51338 (local address 127.6.72.124:42389)
0925 00:47:40.412579 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.412583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.416618 (+ 4035us) server_negotiation.cc:407] Connection header received
0925 00:47:40.417012 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.417015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.417061 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.417110 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.418338 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.419076 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.422611 (+ 3535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.422953 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.423031 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.429180 (+ 6149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.429184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.429186 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.429213 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.430891 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.431425 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.431429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.431431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.431537 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.431842 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.431845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.431847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.432066 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.432205 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.432587 (+ 382us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.432716 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 00:47:40.437522 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.413142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.6.72.126:39591)
0925 00:47:40.413285 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.413291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.418268 (+ 4977us) server_negotiation.cc:407] Connection header received
0925 00:47:40.421139 (+ 2871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.421143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.421207 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.421261 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.423188 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.423948 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.427020 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.427327 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.427411 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.433666 (+ 6255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.433671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.433673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.433701 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.435528 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.436105 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.436109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.436110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.436216 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.436553 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.436556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.436558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.436774 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.436886 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.437246 (+ 360us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.437361 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 00:47:40.438992 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:40.411731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.6.72.125:33767)
0925 00:47:40.411846 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.411850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.412513 (+ 663us) server_negotiation.cc:407] Connection header received
0925 00:47:40.412724 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.412727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.412784 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.412848 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.428062 (+ 15214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.428856 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.429975 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:40.430319 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:40.430386 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:40.435392 (+ 5006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 00:47:40.435396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 00:47:40.435399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 00:47:40.435425 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 00:47:40.437103 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.437729 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.437732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.437734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.437816 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 00:47:40.438094 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 00:47:40.438097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 00:47:40.438099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 00:47:40.438297 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 00:47:40.438398 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:40.438726 (+ 328us) server_negotiation.cc:299] Negotiation successful
0925 00:47:40.438843 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260925 00:47:40.441301 29328 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 00:47:40.441380 29328 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 00:47:40.443384 29174 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [term 1 FOLLOWER]: Refusing update from remote peer a68efade82b64cf3bda17b5a8ff5af81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:40.443385 29250 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [term 1 FOLLOWER]: Refusing update from remote peer a68efade82b64cf3bda17b5a8ff5af81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 00:47:40.444037 29330 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:40.444288 29283 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 00:47:40.448949 29298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [sys.catalog]: SysCatalogTable state changed. Reason: New leader a68efade82b64cf3bda17b5a8ff5af81. Latest consensus state: current_term: 1 leader_uuid: "a68efade82b64cf3bda17b5a8ff5af81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } } }
I20260925 00:47:40.449070 29298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:40.450109 29299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a68efade82b64cf3bda17b5a8ff5af81. Latest consensus state: current_term: 1 leader_uuid: "a68efade82b64cf3bda17b5a8ff5af81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } } }
I20260925 00:47:40.450241 29299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:40.462074 29330 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a68efade82b64cf3bda17b5a8ff5af81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } } }
I20260925 00:47:40.462226 29330 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:40.462853 29283 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a68efade82b64cf3bda17b5a8ff5af81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } } }
I20260925 00:47:40.462965 29283 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81 [sys.catalog]: This master's current role is: LEADER
I20260925 00:47:40.464126 29299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a68efade82b64cf3bda17b5a8ff5af81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } } }
I20260925 00:47:40.464257 29299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:40.465358 29298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a68efade82b64cf3bda17b5a8ff5af81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a68efade82b64cf3bda17b5a8ff5af81" member_type: VOTER last_known_addr { host: "127.6.72.126" port: 39591 } } peers { permanent_uuid: "1c6292626f374eb4af128e0e78922fef" member_type: VOTER last_known_addr { host: "127.6.72.125" port: 33767 } } peers { permanent_uuid: "63a80b356b4f4cceb799c0c7d3361a79" member_type: VOTER last_known_addr { host: "127.6.72.124" port: 42389 } } }
I20260925 00:47:40.465476 29298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef [sys.catalog]: This master's current role is: FOLLOWER
I20260925 00:47:40.466265 29336 mvcc.cc:205] Tried to move back new op lower bound from 7333057578769367040 to 7333057578683715584. Current Snapshot: MvccSnapshot[applied={T|T < 7333057578769367040}]
I20260925 00:47:40.468132 29332 catalog_manager.cc:1347] Generated new cluster ID: 5cf51941b34b40c7a74a66165902c1fb
I20260925 00:47:40.468204 29332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 00:47:40.482220 29332 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 00:47:40.483511 29332 catalog_manager.cc:1504] Loading token signing keys...
I20260925 00:47:40.499272 29332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a68efade82b64cf3bda17b5a8ff5af81: Generated new TSK 0
I20260925 00:47:40.500204 29332 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 00:47:40.534883 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.531481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.6.72.125:33767)
0925 00:47:40.531610 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.531614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.533501 (+ 1887us) server_negotiation.cc:407] Connection header received
0925 00:47:40.533992 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.533996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.534049 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.534101 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.534741 (+ 640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47268: BlockingRecv error: recv got EOF from 127.0.0.1:47268 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260925 00:47:40.536541 29280 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.532210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51344 (local address 127.6.72.124:42389)
0925 00:47:40.532353 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.532357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.535265 (+ 2908us) server_negotiation.cc:407] Connection header received
0925 00:47:40.535755 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.535758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.535809 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.535866 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.536427 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51344: BlockingRecv error: recv got EOF from 127.0.0.1:51344 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260925 00:47:40.537596 29128 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.532893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.6.72.126:39591)
0925 00:47:40.533009 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.533013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.535668 (+ 2655us) server_negotiation.cc:407] Connection header received
0925 00:47:40.536894 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.536897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.536950 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.537001 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.537486 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51088: BlockingRecv error: recv got EOF from 127.0.0.1:51088 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260925 00:47:40.542281 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.540718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47278 (local address 127.6.72.125:33767)
0925 00:47:40.540829 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.540833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.541316 (+ 483us) server_negotiation.cc:407] Connection header received
0925 00:47:40.541535 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.541539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.541616 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.541668 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.542168 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47278: BlockingRecv error: recv got EOF from 127.0.0.1:47278 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260925 00:47:40.547102 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.545627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47290 (local address 127.6.72.125:33767)
0925 00:47:40.545727 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.545731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.546239 (+ 508us) server_negotiation.cc:407] Connection header received
0925 00:47:40.546423 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.546426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.546475 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.546524 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.547000 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47290: BlockingRecv error: recv got EOF from 127.0.0.1:47290 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260925 00:47:40.552659 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.551075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47296 (local address 127.6.72.125:33767)
0925 00:47:40.551173 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.551177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.551687 (+ 510us) server_negotiation.cc:407] Connection header received
0925 00:47:40.551873 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.551876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.551928 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.551978 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.552561 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47296: BlockingRecv error: recv got EOF from 127.0.0.1:47296 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260925 00:47:40.560410 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.558737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47306 (local address 127.6.72.125:33767)
0925 00:47:40.558882 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.558886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.559414 (+ 528us) server_negotiation.cc:407] Connection header received
0925 00:47:40.559603 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.559606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.559660 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.559710 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.560280 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47306: BlockingRecv error: recv got EOF from 127.0.0.1:47306 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260925 00:47:40.572408 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.570632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47322 (local address 127.6.72.125:33767)
0925 00:47:40.570754 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.570759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.571346 (+ 587us) server_negotiation.cc:407] Connection header received
0925 00:47:40.571558 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.571563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.571620 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.571678 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.572288 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47322: BlockingRecv error: recv got EOF from 127.0.0.1:47322 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260925 00:47:40.593694 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.591829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.6.72.125:33767)
0925 00:47:40.591952 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.591957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.592609 (+ 652us) server_negotiation.cc:407] Connection header received
0925 00:47:40.592826 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.592831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.592892 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.592946 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.593573 (+ 627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47324: BlockingRecv error: recv got EOF from 127.0.0.1:47324 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260925 00:47:40.629693 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.628331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47330 (local address 127.6.72.125:33767)
0925 00:47:40.628464 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.628469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.628733 (+ 264us) server_negotiation.cc:407] Connection header received
0925 00:47:40.628920 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.628923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.628973 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.629064 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.629525 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47330: BlockingRecv error: recv got EOF from 127.0.0.1:47330 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260925 00:47:40.697139 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.695873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47338 (local address 127.6.72.125:33767)
0925 00:47:40.696034 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.696039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.696173 (+ 134us) server_negotiation.cc:407] Connection header received
0925 00:47:40.696390 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.696393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.696442 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.696536 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.696993 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47338: BlockingRecv error: recv got EOF from 127.0.0.1:47338 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260925 00:47:40.828763 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:40.827464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.6.72.125:33767)
0925 00:47:40.827595 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:40.827601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:40.827765 (+ 164us) server_negotiation.cc:407] Connection header received
0925 00:47:40.827965 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:40.827969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:40.828031 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:40.828122 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:40.828638 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47344: BlockingRecv error: recv got EOF from 127.0.0.1:47344 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 00:47:41.088169 29206 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 00:47:41.087072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.6.72.125:33767)
0925 00:47:41.087205 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:41.087208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:41.087231 (+ 23us) server_negotiation.cc:407] Connection header received
0925 00:47:41.087346 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:41.087349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:41.087396 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:41.087479 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 00:47:41.088023 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47352: BlockingRecv error: recv got EOF from 127.0.0.1:47352 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 00:47:41.417896 29325 catalog_manager.cc:1259] Loaded cluster ID: 5cf51941b34b40c7a74a66165902c1fb
I20260925 00:47:41.418013 29325 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79: loading cluster ID for follower catalog manager: success
I20260925 00:47:41.419668 29325 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79: acquiring CA information for follower catalog manager: success
I20260925 00:47:41.420900 29325 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 63a80b356b4f4cceb799c0c7d3361a79: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 00:47:41.442649 29328 catalog_manager.cc:1259] Loaded cluster ID: 5cf51941b34b40c7a74a66165902c1fb
I20260925 00:47:41.442765 29328 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef: loading cluster ID for follower catalog manager: success
I20260925 00:47:41.444531 29328 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef: acquiring CA information for follower catalog manager: success
I20260925 00:47:41.445662 29328 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1c6292626f374eb4af128e0e78922fef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 00:47:41.607246 29406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:41.602981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.6.72.125:33767)
0925 00:47:41.603281 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:41.603286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:41.603302 (+ 16us) server_negotiation.cc:407] Connection header received
0925 00:47:41.603381 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:41.603383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:41.603513 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:41.603622 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:41.604013 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:41.604591 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:41.605586 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:41.605838 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:41.605965 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:41.606324 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:41.606798 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:41.606952 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:41.606994 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 00:47:41.607060 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":101,"threads_started":1}
I20260925 00:47:41.615619 29412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:41.611510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.6.72.124:42389)
0925 00:47:41.611778 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:41.611782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:41.611797 (+ 15us) server_negotiation.cc:407] Connection header received
0925 00:47:41.612021 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:41.612024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:41.612154 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:41.612353 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:41.612622 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:41.613176 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:41.614036 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:41.614298 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:41.614396 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:41.614763 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:41.615251 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:41.615357 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:41.615424 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 00:47:41.615481 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":92,"threads_started":1}
I20260925 00:47:41.623554 29412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:41.619637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.6.72.124:42389)
0925 00:47:41.619758 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:41.619762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:41.620016 (+ 254us) server_negotiation.cc:407] Connection header received
0925 00:47:41.620056 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:41.620058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:41.620129 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:41.620201 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:41.620491 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:41.621210 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:41.622353 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:41.622570 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:41.622691 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:41.623030 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:41.623117 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:41.623190 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:41.623333 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 00:47:41.623385 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 00:47:41.623595 29406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:41.619061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47376 (local address 127.6.72.125:33767)
0925 00:47:41.619219 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:41.619223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:41.619554 (+ 331us) server_negotiation.cc:407] Connection header received
0925 00:47:41.619695 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:41.619700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:41.619764 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:41.619847 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:41.620128 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:41.620939 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:41.621951 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:41.622196 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:41.622314 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:41.623067 (+ 753us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:41.623151 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:41.623238 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:41.623378 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 00:47:41.623437 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 00:47:41.624032 29419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 00:47:41.619008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51090 (local address 127.6.72.126:39591)
0925 00:47:41.619288 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0925 00:47:41.619292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 00:47:41.619408 (+ 116us) server_negotiation.cc:407] Connection header received
0925 00:47:41.619571 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 00:47:41.619574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 00:47:41.619750 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 00:47:41.619907 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 00:47:41.620296 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:41.620905 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:41.622297 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 00:47:41.622551 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 00:47:41.622626 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 00:47:41.623032 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 00:47:41.623390 (+ 358us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 00:47:41.623472 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 00:47:41.623843 (+ 371us) server_negotiation.cc:299] Negotiation successful
0925 00:47:41.623905 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":78,"threads_started":1}
I20260925 00:47:41.627225 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 29057
I20260925 00:47:41.633771 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 29132
I20260925 00:47:41.639416 6433 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklhk22w/build/debug/bin/kudu with pid 29208
2026-09-25T00:47:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3088 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6357 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (290276 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST